Lab 5.1

Shrikaa

BusinessAnalyticsDataAnalysisDecisionMaking7E.pdf

Home >Computer Science homework help >Lab 5.1

BUSINESS ANALYTICS

Data Analysis and Decision Making

SEVENTH EDITION

S. Christian Albright Kelly School of Business,

Indiana University, Emeritus

Wayne L. Winston Kelly School of Business,

Indiana University, Emeritus

Australia • Brazil • Mexico • Singapore • United Kingdom • United States

09953_fm_ptg01_i-xxiv.indd 1 04/03/19 5:53 PM

Copyright 2020 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to remove additional content at any time if subsequent rights restrictions require it.

This is an electronic version of the print textbook. Due to electronic rights restrictions, some third party content may be suppressed. Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. The publisher reserves the right to remove content from this title at any time if subsequent rights restrictions require it. For valuable information on pricing, previous editions, changes to current editions, and alternate formats, please visit www.cengage.com/highered to search by ISBN#, author, title, or keyword for materials in your areas of interest.

Important Notice: Media content referenced within the product description or the product text may not be available in the eBook version.

Unless otherwise noted, all content is © Cengage

ALL RIGHTS RESERVED. No part of this work covered by the copyright herein may be reproduced or distributed in any form or by any means, except as permitted by U.S. copyright law, without the prior written permission of the copyright owner.

For product information and technology assistance, contact us at Cengage

Customer & Sales Support, 1-800-354-9706 or support.cengage.com.

For permission to use material from this text or product,

submit all requests online at www.cengage.com/permissions.

Library of Congress Control Number: 2019935644

Student Edition: ISBN: 978-0-357-10995-3

Loose-leaf Edition: ISBN: 978-0-357-10996-0

Cengage 20 Channel Center Street Boston, MA 02210 USA

Cengage is a leading provider of customized learning solutions with employees residing in nearly 40 different countries and sales in more than 125 countries around the world. Find your local representative at www. cengage.com.

Cengage products are represented in Canada by Nelson Education, Ltd.

To learn more about Cengage platforms and services, register or access your online learning solution, or purchase materials for your course, visit www.cengage.com.

Business Analytics: Data Analysis and Decision Making, 7e

S. Christian Albright and Wayne L. Winston

Senior Vice President, Higher Ed Product, Content, and Market Development: Erin Joyner

Product Director: Jason Fremder

Senior Product Manager: Aaron Arnsparger

Senior Learning Designer: Brandon Foltz

Content Manager: Conor Allen

Digital Delivery Lead: Mark Hopkinson

Product Assistant: Christian Wood

Marketing Manager: Chris Walz

Production Service: Lumina Datamatics Ltd.

Designer, Creative Studio: Erin Griffin

Text Designer: Erin Griffin

Cover Designer: Erin Griffin

Cover Image(s): Ron Dale/ShutterStock.com

Intellectual Property Analyst: Reba Frederics

Intellectual Property Project Manager: Nick

Barrows

Printed in the United States of America Print Number: 01 Print Year: 2019

09953_fm_ptg01_i-xxiv.indd 2 05/03/19 9:35 PM

To my wonderful wife Mary—my best friend and companion; and to Sam, Lindsay,

Teddy, and Archie S.C.A

To my wonderful family W.L.W.

09953_fm_ptg01_i-xxiv.indd 3 04/03/19 5:53 PM

S. Christian Albright got his B.S. degree in Mathematics from Stanford in 1968 and his PhD in Operations Research from Stanford in 1972. He taught in the Operations & Decision Technologies Department in the Kelley School of Business at Indiana University (IU) for close to 40 years, before retiring from teaching in 2011. While at IU, he taught courses in management science, computer simulation, statistics, and computer programming to all levels of business students, including undergraduates, MBAs, and doctoral students. In addition, he taught simula- tion modeling at General Motors and Whirlpool, and he taught database analysis for the Army. He published over 20 articles in leading operations research journals in the area of applied probabil- ity, and he has authored the books Statistics for Business and Economics, Practical Management Science, Spreadsheet Modeling and Applications, Data Analysis for Managers, and VBA for Mod- elers. He worked for several years after “retirement” with the Palisade Corporation developing training materials for its software products, he has developed a commercial version of his Excel® tutorial, called ExcelNow!, and he continues to revise his textbooks.

On the personal side, Chris has been married for 47 years to his wonderful wife, Mary, who retired several years ago after teaching 7th grade English for 30 years. They have one son, Sam, who lives in Philadelphia with his wife Lindsay and their two sons, Teddy and Archie. Chris has many interests outside the academic area. They include activities with his family, traveling with Mary, going to cultural events, power walking while listening to books on his iPod, and reading. And although he earns his livelihood from quantitative methods, his real passion is for playing classical piano music.

Wayne L. Winston taught in the Operations & Decision Technologies Department in the Kelley School of Business at Indiana University for close to 40 before retiring a few years ago. Wayne received his B.S. degree in Mathematics from MIT and his PhD in Operations Research from Yale. He has written the successful textbooks Operations Research: Applications and Algorithms, Mathematical Programming: Applications and Algorithms, Simulation Modeling Using @RISK, Practical Management Science, Data Analysis and Decision Making, Financial Models Using Simulation and Optimization, and Mathletics. Wayne has published more than 20 articles in lead- ing journals and has won many teaching awards, including the school-wide MBA award four times. He has taught classes at Microsoft, GM, Ford, Eli Lilly, Bristol-Myers Squibb, Arthur Andersen, Roche, PricewaterhouseCoopers, and NCR, and in “retirement,” he is currently teach- ing several courses at the University of Houston. His current interest is showing how spread- sheet models can be used to solve business problems in all disciplines, particularly in finance and marketing.

Wayne enjoys swimming and basketball, and his passion for trivia won him an appearance several years ago on the television game show Jeopardy!, where he won two games. He is married to the lovely and talented Vivian. They have two children, Gregory and Jennifer.

ABOUT THE AUTHORS

09953_fm_ptg01_i-xxiv.indd 4 04/03/19 5:53 PM

BRIEF CONTENTS Preface xvi

1 Introduction to Business Analytics 1

PART 1 Data Analysis 37 2 Describing the Distribution of a Variable 38 3 Finding Relationships among Variables 84 4 Business Intelligence (BI) Tools for Data Analysis 132

PART 2 Probability and Decision Making under Uncertainty 183 5 Probability and Probability Distributions 184 6 Decision Making under Uncertainty 242

PART 3 Statistical Inference 293 7 Sampling and Sampling Distributions 294 8 Confidence Interval Estimation 323 9 Hypothesis Testing 368

PART 4 Regression Analysis and Time Series Forecasting 411 10 Regression Analysis: Estimating Relationships 412 11 Regression Analysis: Statistical Inference 472 12 Time Series Analysis and Forecasting 523

PART 5 Optimization and Simulation Modeling 575 13 Introduction to Optimization Modeling 576 14 Optimization Models 630 15 Introduction to Simulation Modeling 717 16 Simulation Models 779

PART 6 Advanced Data Analysis 837 17 Data Mining 838 18 Analysis of Variance and Experimental Design (MindTap Reader only) 19 Statistical Process Control (MindTap Reader only) APPENDIX A: Quantitative Reporting (MindTap Reader only)

References 873

Index 875

09953_fm_ptg01_i-xxiv.indd 5 04/03/19 5:54 PM

CONTENTS Preface xvi

1 Introduction to Business Analytics 1

1-1 Introduction 3

1-2 Overview of the Book 4

1-2a The Methods 4 1-2b The Software 6

1-3 Introduction to Spreadsheet Modeling 8

1-3a Basic Spreadsheet Modeling: Concepts and Best Practices 9 1-3b Cost Projections 12 1-3c Breakeven Analysis 15 1-3d Ordering with Quantity Discounts and Demand Uncertainty 20 1-3e Estimating the Relationship between Price and Demand 24 1-3f Decisions Involving the Time Value of Money 29

1-4 Conclusion 33

PART 1 Data Analysis 37

2 Describing the Distribution of a Variable 38

2-1 Introduction 39

2-2 Basic Concepts 41

2-2a Populations and Samples 41 2-2b Data Sets, Variables, and Observations 41 2-2c Data Types 42

2-3 Summarizing Categorical Variables 45

2-4 Summarizing Numeric Variables 49

2-4a Numeric Summary Measures 49 2-4b Charts for Numeric Variables 57

2-5 Time Series Data 62

2-6 Outliers and Missing Values 69

2-7 Excel Tables for Filtering, Sorting, and Summarizing 71

2-8 Conclusion 77

Appendix: Introduction to StatTools 83

3 Finding Relationships among Variables 84

3-1 Introduction 85

3-2 Relationships among Categorical Variables 86

09953_fm_ptg01_i-xxiv.indd 6 04/03/19 5:54 PM

C O N T E N T S v i i

3-3 Relationships among Categorical Variables and a Numeric Variable 89

3-4 Relationships among Numeric Variables 96

3-4a Scatterplots 96 3-4b Correlation and Covariance 101

3-5 Pivot Tables 106

3-6 Conclusion 126

Appendix: Using StatTools to Find Relationships 131

4 Business Intelligence (BI) Tools for Data Analysis 132

4-1 Introduction 133

4-2 Importing Data into Excel with Power Query 134

4-2a Introduction to Relational Databases 134 4-2b Excel’s Data Model 139 4-2c Creating and Editing Queries 146

4-3 Data Analysis with Power Pivot 152

4-3a Basing Pivot Tables on a Data Model 154 4-3b Calculated Columns, Measures, and the DAX Language 154

4-4 Data Visualization with Tableau Public 162

4-5 Data Cleansing 172

4-6 Conclusion 178

PART 2 Probability and Decision Making under Uncertainty 183

5 Probability and Probability Distributions 184

5-1 Introduction 185

5-2 Probability Essentials 186

5-2a Rule of Complements 187 5-2b Addition Rule 187 5-2c Conditional Probability and the Multiplication Rule 188 5-2d Probabilistic Independence 190 5-2e Equally Likely Events 191 5-2f Subjective Versus Objective Probabilities 192

5-3 Probability Distribution of a Random Variable 194

5-3a Summary Measures of a Probability Distribution 195 5-3b Conditional Mean and Variance 198

5-4 The Normal Distribution 200

5-4a Continuous Distributions and Density Functions 200 5-4b The Normal Density Function 201 5-4c Standardizing: Z-Values 202 5-4d Normal Tables and Z-Values 204

09953_fm_ptg01_i-xxiv.indd 7 04/03/19 5:54 PM

v i i i C O N T E N T S

5-4e Normal Calculations in Excel 205 5-4f Empirical Rules Revisited 208 5-4g Weighted Sums of Normal Random Variables 208 5-4h Normal Distribution Examples 209

5-5 The Binomial Distribution 214

5-5a Mean and Standard Deviation of the Binomial Distribution 217 5-5b The Binomial Distribution in the Context of Sampling 217 5-5c The Normal Approximation to the Binomial 218 5-5d Binomial Distribution Examples 219

5-6 The Poisson and Exponential Distributions 226

5-6a The Poisson Distribution 227 5-6b The Exponential Distribution 229

5-7 Conclusion 231

6 Decision Making under Uncertainty 242

6-1 Introduction 243

6-2 Elements of Decision Analysis 244

6-3 EMV and Decision Trees 247

6-4 One-Stage Decision Problems 251

6-5 The PrecisionTree Add-In 254

6-6 Multistage Decision Problems 257

6.6a Bayes’ Rule 262 6-6b The Value of Information 267 6-6c Sensitivity Analysis 270

6-7 The Role of Risk Aversion 274

6-7a Utility Functions 275 6-7b Exponential Utility 275 6-7c Certainty Equivalents 278 6-7d Is Expected Utility Maximization Used? 279

6-8 Conclusion 280

PART 3 Statistical Inference 293

7 Sampling and Sampling Distributions 294

7-1 Introduction 295

7-2 Sampling Terminology 295

7-3 Methods for Selecting Random Samples 297

7-3a Simple Random Sampling 297 7-3b Systematic Sampling 301 7-3c Stratified Sampling 301

09953_fm_ptg01_i-xxiv.indd 8 04/03/19 5:54 PM

C O N T E N T S i x

7-3d Cluster Sampling 303 7-3e Multistage Sampling 303

7-4 Introduction to Estimation 305

7-4a Sources of Estimation Error 305 7-4b Key Terms in Sampling 306 7-4c Sampling Distribution of the Sample Mean 307 7-4d The Central Limit Theorem 312 7-4e Sample Size Selection 317 7-4f Summary of Key Ideas in Simple Random Sampling 318

7-5 Conclusion 320

8 Confidence Interval Estimation 323

8-1 Introduction 323

8-2 Sampling Distributions 325

8-2a The t Distribution 326 8-2b Other Sampling Distributions 327

8-3 Confidence Interval for a Mean 328

8-4 Confidence Interval for a Total 333

8-5 Confidence Interval for a Proportion 336

8-6 Confidence Interval for a Standard Deviation 340

8-7 Confidence Interval for the Difference between Means 343

8-7a Independent Samples 344 8-7b Paired Samples 346

8-8 Confidence Interval for the Difference between Proportions 348

8-9 Sample Size Selection 351

8-10 Conclusion 358

9 Hypothesis Testing 368

9-1 Introduction 369

9-2 Concepts in Hypothesis Testing 370

9-2a Null and Alternative Hypotheses 370 9-2b One-Tailed Versus Two-Tailed Tests 371 9-2c Types of Errors 372 9-2d Significance Level and Rejection Region 372 9-2e Significance from p-values 373 9-2f Type II Errors and Power 375 9-2g Hypothesis Tests and Confidence Intervals 375 9-2h Practical Versus Statistical Significance 375

9-3 Hypothesis Tests for a Population Mean 376

9-4 Hypothesis Tests for Other Parameters 380

09953_fm_ptg01_i-xxiv.indd 9 04/03/19 5:54 PM

x C O N T E N T S

9-4a Hypothesis Test for a Population Proportion 380 9-4b Hypothesis Tests for Difference between Population Means 382 9-4c Hypothesis Test for Equal Population Variances 388 9-4d Hypothesis Test for Difference between Population Proportions 388

9-5 Tests for Normality 395

9-6 Chi-Square Test for Independence 401

9-7 Conclusion 404

PART 4 Regression Analysis and Time Series Forecasting 411

10 Regression Analysis: Estimating Relationships 412

10-1 Introduction 413

10-2 Scatterplots: Graphing Relationships 415

10-3 Correlations: Indicators of Linear Relationships 422

10-4 Simple Linear Regression 424

10-4a Least Squares Estimation 424 10-4b Standard Error of Estimate 431 10-4c R-Square 432

10-5 Multiple Regression 435

10-5a Interpretation of Regression Coefficients 436 10-5b Interpretation of Standard Error of Estimate and R-Square 439

10-6 Modeling Possibilities 442

10-6a Dummy Variables 442 10-6b Interaction Variables 448 10-6c Nonlinear Transformations 452

10-7 Validation of the Fit 461

10-8 Conclusion 463

11 Regression Analysis: Statistical Inference 472

11-1 Introduction 473

11-2 The Statistical Model 474

11-3 Inferences About the Regression Coefficients 477

11-3a Sampling Distribution of the Regression Coefficients 478 11-3b Hypothesis Tests for the Regression Coefficients and p-Values 480 11-3c A Test for the Overall Fit: The ANOVA Table 481

11-4 Multicollinearity 485

11-5 Include/Exclude Decisions 489

11-6 Stepwise Regression 494

11-7 Outliers 499

09953_fm_ptg01_i-xxiv.indd 10 04/03/19 5:54 PM

C O N T E N T S x i

11-8 Violations of Regression Assumptions 504

11-8a Nonconstant Error Variance 504 11-8b Nonnormality of Residuals 504 11-8c Autocorrelated Residuals 505

11-9 Prediction 507

11-10 Conclusion 512

12 Time Series Analysis and Forecasting 523

12-1 Introduction 524

12-2 Forecasting Methods: An Overview 525

12-2a Extrapolation Models 525 12-2b Econometric Models 526 12-2c Combining Forecasts 526 12-2d Components of Time Series Data 527 12-2e Measures of Accuracy 529

12-3 Testing for Randomness 531

12-3a The Runs Test 534 12-3b Autocorrelation 535

12-4 Regression-Based Trend Models 539

12-4a Linear Trend 539 12-4b Exponential Trend 541

12-5 The Random Walk Model 544

12-6 Moving Averages Forecasts 547

12-7 Exponential Smoothing Forecasts 551

12-7a Simple Exponential Smoothing 552 12-7b Holt’s Model for Trend 556

12-8 Seasonal Models 560

12-8a Winters’ Exponential Smoothing Model 561 12-8b Deseasonalizing: The Ratio-to-Moving-Averages Method 564 12-8c Estimating Seasonality with Regression 565

12-9 Conclusion 569

PART 5 Optimization and Simulation Modeling 575

13 Introduction to Optimization Modeling 576

13-1 Introduction 577

13-2 Introduction to Optimization 577

13-3 A Two-Variable Product Mix Model 579

09953_fm_ptg01_i-xxiv.indd 11 04/03/19 5:54 PM

x i i C O N T E N T S

13-4 Sensitivity Analysis 590

13-4a Solver’s Sensitivity Report 590 13-4b SolverTable Add-In 593 13-4c A Comparison of Solver’s Sensitivity Report and SolverTable 599

13-5 Properties of Linear Models 600

13-6 Infeasibility and Unboundedness 602

13-7 A Larger Product Mix Model 604

13-8 A Multiperiod Production Model 612

13-9 A Comparison of Algebraic and Spreadsheet Models 619

13-10 A Decision Support System 620

13-11 Conclusion 622

14 Optimization Models 630

14-1 Introduction 631

14-2 Employee Scheduling Models 632

14-3 Blending Models 638

14-4 Logistics Models 644

14-4a Transportation Models 644 14-4b More General Logistics Models 651

14-5 Aggregate Planning Models 659

14-6 Financial Models 667

14-7 Integer Optimization Models 677

14-7a Capital Budgeting Models 678 14-7b Fixed-Cost Models 682 14-7c Set-Covering Models 689

14-8 Nonlinear Optimization Models 695

14-8a Difficult Issues in Nonlinear Optimization 695 14-8b Managerial Economics Models 696 14-8c Portfolio Optimization Models 700

14-9 Conclusion 708

15 Introduction to Simulation Modeling 717

15-1 Introduction 718

15-2 Probability Distributions for Input Variables 720

15-2a Types of Probability Distributions 721 15-2b Common Probability Distributions 724 15-2c Using @RISK to Explore Probability Distributions 728

15-3 Simulation and the Flaw of Averages 736

15-4 Simulation with Built-in Excel Tools 738

09953_fm_ptg01_i-xxiv.indd 12 04/03/19 5:54 PM

C O N T E N T S x i i i

15-5 Simulation with @RISK 747

15-5a @RISK Features 748 15-5b Loading @RISK 748 15-5c @RISK Models with a Single Random Input 749 15-5d Some Limitations of @RISK 758 15-5e @RISK Models with Several Random Inputs 758

15-6 The Effects of Input Distributions on Results 763

15-6a Effect of the Shape of the Input Distribution(s) 763 15-6b Effect of Correlated Inputs 766

15-7 Conclusion 771

16 Simulation Models 779

16-1 Introduction 780

16-2 Operations Models 780

16-2a Bidding for Contracts 780 16-2b Warranty Costs 784 16-2c Drug Production with Uncertain Yield 789

16-3 Financial Models 794

16-3a Financial Planning Models 795 16-3b Cash Balance Models 799 16-3c Investment Models 803

16-4 Marketing Models 810

16-4a Customer Loyalty Models 810 16-4b Marketing and Sales Models 817

16-5 Simulating Games of Chance 823

16-5a Simulating the Game of Craps 823 16-5b Simulating the NCAA Basketball Tournament 825

16-6 Conclusion 828

PART 6 Advanced Data Analysis 837

17 Data Mining 838

17-1 Introduction 839

17-2 Classification Methods 840

17-2a Logistic Regression 841 17-2b Neural Networks 846 17-2c Naïve Bayes 851 17-2d Classification Trees 854 17-2e Measures of Classification Accuracy 855 17-2f Classification with Rare Events 857

09953_fm_ptg01_i-xxiv.indd 13 04/03/19 5:54 PM

x i v C O N T E N T S

17-3 Clustering Methods 860

17-4 Conclusion 870

18 Analysis of Variance and Experimental Design (MindTap Reader only)

18-1 Introduction 18-2

18-2 One-Way ANOVA 18-5

18-2a The Equal-Means Test 18-5 18-2b Confidence Intervals for Differences Between Means 18-7 18-2c Using a Logarithmic Transformation 18-11

18-3 Using Regression to Perform ANOVA 18-15

18-4 The Multiple Comparison Problem 18-18

18-5 Two-Way ANOVA 18-22

18-5a Confidence Intervals for Contrasts 18-28 18-5b Assumptions of Two-Way ANOVA 18-30

18-6 More About Experimental Design 18-32

18-6a Randomization 18-32 18-6b Blocking 18-35 18-6c Incomplete Designs 18-38

18-7 Conclusion 18-40

19 Statistical Process Control (MindTap Reader only)

19-1 Introduction 19-2

19-2 Deming’s 14 Points 19-3

19-3 Introduction to Control Charts 19-6

19-4 Control Charts for Variables 19-8

19-4a Control Charts and Hypothesis Testing 19-13 19-4b Other Out-of-Control Indications 19-15 19-4c Rational Subsamples 19-16 19-4d Deming’s Funnel Experiment and Tampering 19-18 19-4e Control Charts in the Service Industry 19-22

19-5 Control Charts for Attributes 19-26

19-5a P Charts 19-26 19-5b Deming’s Red Bead Experiment 19-29

19-6 Process Capability 19-33

19-6a Process Capability Indexes 19-35 19-6b More on Motorola and 6-Sigma 19-40

19-7 Conclusion 19-43

09953_fm_ptg01_i-xxiv.indd 14 04/03/19 5:54 PM

C O N T E N T S x v

APPENDIX A: Quantitative Reporting (MindTap Reader only)

A-1 Introduction A-1

A-2 Suggestions for Good Quantitative Reporting A-2

A-2a Planning A-2 A-2b Developing a Report A-3 A-2c Be Clear A-4 A-2d Be Concise A-4 A-2e Be Precise A-5

A-3 Examples of Quantitative Reports A-6

A-4 Conclusion A-16

References 873

Index 875

09953_fm_ptg01_i-xxiv.indd 15 04/03/19 5:54 PM

PREFACE With today’s technology, companies are able to collect tremendous amounts of data with relative ease. Indeed, many com- panies now have more data than they can handle. However, before the data can be useful, they must be analyzed for trends, patterns, and relationships. This book illustrates in a practical way a variety of methods, from simple to complex, to help you analyze data sets and uncover important information. In many business contexts, data analysis is only the first step in the solution of a problem. Acting on the solution and the information it provides to make good decisions is a critical next step. Therefore, there is a heavy emphasis throughout this book on analytical methods that are useful in decision making. The meth- ods vary considerably, but the objective is always the same—to equip you with decision-making tools that you can apply in your business careers.

We recognize that the majority of students in this type of course are not majoring in a quantitative area. They are typically business majors in finance, marketing, operations management, or some other business discipline who will need to analyze data and make quantitative-based decisions in their jobs. We offer a hands-on, example-based approach and introduce fundamental concepts as they are needed. Our vehicle is spreadsheet software—specifically, Microsoft Excel®. This is a package that most students already know and will almost surely use in their careers. Our MBA students at Indiana University have been so turned on by the required course that is based on this book that almost all of them (mostly finance and marketing majors) have taken at least one of our follow-up elective courses in spreadsheet modeling. We are convinced that students see value in quantitative analysis when the course is taught in a practical and example-based approach.

Rationale for Writing This Book Business Analytics: Data Analysis and Decision Making is different from other textbooks written for statistics and management science. Our rationale for writing this book is based on four fundamental objectives.

• Integrated coverage and applications. The book provides a unified approach to business-related problems by integrat- ing methods and applications that have been traditionally taught in separate courses, specifically statistics and manage- ment science.

• Practical in approach. The book emphasizes realistic business examples and the processes managers actually use to analyze business problems. The emphasis is not on abstract theory or computational methods.

• Spreadsheet-based teaching. The book provides students with the skills to analyze business problems with tools they have access to and will use in their careers. To this end, we have adopted Excel and commercial spreadsheet add-ins.

• Latest tools. This is not a static field. The software keeps changing, and even the mathematical algorithms behind the software continue to evolve. Each edition of this book has presented the most recent tools in Excel and the accompanying Excel add-ins, and the current edition is no exception.

Integrated Coverage and Applications In the past, many business schools have offered a required statistics course, a required decision-making course, and a required management science course—or some subset of these. The current trend, however, is to have only one required course that cov- ers the basics of statistics, some regression analysis, some decision making under uncertainty, some linear programming, some simulation, and some advanced data analysis methods. Essentially, faculty in the quantitative area get one opportunity to teach all business students, so we attempt to cover a variety of useful quantitative methods. We are not necessarily arguing that this trend is ideal, but rather that it is a reflection of the reality at our university and, we suspect, at many others. After several years of teaching this course, we have found it to be a great opportunity to attract students to the subject and to more advanced study.

The book is also integrative in another important aspect. It not only integrates a number of analytical methods, but it also applies them to a wide variety of business problems—that is, it analyzes realistic examples from many business disciplines. We include examples, problems, and cases that deal with portfolio optimization, workforce scheduling, market share analysis, capital budgeting, new product analysis, and many others.

09953_fm_ptg01_i-xxiv.indd 16 04/03/19 5:54 PM

P R E F A C E x v i i

Practical in Approach This book has been designed to be very example-based and practical. We strongly believe that students learn best by working through examples, and they appreciate the material most when the examples are realistic and interesting. Therefore, our approach in the book differs in two important ways from many competitors. First, there is just enough conceptual development to give students an understanding and appreciation for the issues raised in the examples. We often introduce important concepts, such as standard deviation as a measure of variability, in the context of examples rather than discussing them in the abstract. Our experience is that students gain greater intuition and understanding of the concepts and applications through this approach.

Second, we place virtually no emphasis on hand calculations. We believe it is more important for students to understand why they are conducting an analysis and to interpret the results than to emphasize the tedious calculations associated with many analytical techniques. Therefore, we illustrate how powerful software can be used to create graphical and numerical outputs in a matter of seconds, freeing the rest of the time for in-depth interpretation of the results, sensitivity analysis, and alternative modeling approaches.

Spreadsheet-based Teaching We are strongly committed to teaching spreadsheet-based, example-driven courses, regardless of whether the basic area is data analysis or management science. We have found tremendous enthusiasm for this approach, both from students and from faculty around the world who have used our books. Students learn and remember more, and they appreciate the material more. In addition, instructors typically enjoy teaching more, and they usually receive immediate reinforcement through better teaching evaluations. We were among the first to move to spreadsheet-based teaching about two decades ago, and we have never regret- ted the move.

What We Hope to Accomplish in This Book Condensing the ideas in the previous paragraphs, we hope to:

• continue to make quantitative courses attractive to a wide audience by making these topics real, accessible, and interesting;

• give students plenty of hands-on experience with real problems and challenge them to develop their intuition, logic, and problem-solving skills;

• expose students to real problems in many business disciplines and show them how these problems can be analyzed with quantitative methods; and

• develop spreadsheet skills, including experience with powerful spreadsheet add-ins, that add immediate value to stu- dents’ other courses and for their future careers.

New in the Seventh Edition There are several important changes in this edition.

• New introductory material on Excel: Chapter 1 now includes an introductory section on spreadsheet modeling. This provides business examples for getting students up to speed in Excel and covers such Excel tools as IF and VLOOKUP functions, data tables, goal seek, range names, and more.

• Reorganization of probability chapters: Chapter 4, Probability and Probability Distributions, and Chapter 5, Normal, Binomial, Poisson, and Exponential Distributions, have been shortened slightly and combined into a single Chapter 5, Probability and Probability Distributions. This created space for the new Chapter 4 discussed next.

• New material on “Power BI” tools and data visualization: The previous chapters on Data Mining and Importing Data into Excel have been reorganized and rewritten to include an increased focus on the tools commonly included under the Business Analytics umbrella. There is now a new Chapter 4, Business Intelligence Tools for Data Analysis, which includes Excel’s Power Query tools for importing data into Excel, Excel’s Power Pivot add-in (and the DAX language) for even more powerful data analysis with pivot tables, and Tableau Public for data visualization. The old online Chapter 18, Importing Data into Excel, has been eliminated, and its material has been moved to this new Chapter 4.

09953_fm_ptg01_i-xxiv.indd 17 04/03/19 5:54 PM

x v i i i P R E F A C E

• Updated for Office 365, Windows or Mac: The 7th Edition is completely compatible with the latest version of Excel, and all screenshots in the book are from the latest version. However, because the changes from previous versions are not that extensive for Business Analytics purposes, the 7th Edition also works well even if you are still using Microsoft Office 2013, 2010, or 2007. Also, recognizing that many students are now using Macs, we have attempted to make the material compatible with Excel for Mac whenever possible.

• Updated Problems: Numerous problems have been modified to include the most updated data available. In addition, the DADM 7e Problem Database.xlsx file provides instructors with an entire database of problems. This file indicates the context of each of the problems and shows the correspondence between problems in this edition and problems in the previous edition.

• Less emphasis on add-ins (when possible): There is more emphasis in this edition on implementing spreadsheet calculations, especially statistical calculations, with built-in Excel tools rather than with add-ins. For example, there is no reliance on Palisade’s StatTools add-in in the descriptive statistics chapters 2 and 3 or in the confidence interval and hypothesis testing chapters 8 and 9. Nevertheless, Palisade’s add-ins are still relied on in chapters where they are really needed: PrecisionTree for decision trees in Chapter 6; StatTools for regression and time series analysis in Chapters 10, 11, and 12; @RISK for simulation in Chapters 15 and 16; and StatTools and NeuralTools for logistic regression and neu- ral networks in Chapter 17.

• New optional add-in: Although it is not an “official” part of the book, Albright wrote a DADM_Tools add-in for Excel (Windows or Mac), with tools for creating summary stats, histograms, correlations and scatterplots, regression, time series analysis, decision trees, and simulation. This add-in provides a “lighter” alternative to the Palisade add-ins and is freely available at https://kelley.iu.edu/albrightbooks/free_downloads.htm.

Software This book is based entirely on Microsoft Excel, the spreadsheet package that has become the standard analytical tool in busi- ness. Excel is an extremely powerful package, and one of our goals is to convert casual users into power users who can take full advantage of its features. If you learn no more than this, you will be acquiring a valuable skill for the business world. However, Excel has some limitations. Therefore, this book relies on several Excel add-ins to enhance Excel’s capabilities. As a group, these add-ins comprise what is arguably the most impressive assortment of spreadsheet-based software accompanying any book on the market.

DecisionTools® Suite Add-in The textbook website for Business Analytics: Data Analysis and Decision Making provides a link to the powerful DecisionTools® Suite by Palisade Corporation. This suite includes seven separate add-ins:

• @RISK, an add-in for simulation

• StatTools, an add-in for statistical data analysis

• PrecisionTree, a graphical-based add-in for creating and analyzing decision trees

• TopRank, an add-in for performing what-if analyses

• NeuralTools®, an add-in for estimating complex, nonlinear relationships

• EvolverTM, an add-in for performing optimization (an alternative to Excel’s Solver)

• BigPicture, a smart drawing add-in, useful for depicting model relationships

We use @RISK and PrecisionTree extensively in the chapters on simulation and decision making under uncertainty, and we use StatTools as necessary in the data analysis chapters. We also use BigPicture in the optimization and simulation chapters to provide a “bridge” between a problem statement and an eventual spreadsheet model.

Online access to the DecisionTools Suite, available with new copies of the book and for MindTap adopters, is an academic version, slightly scaled down from the professional version that sells for hundreds of dollars and is used by many leading companies. It functions for one year when properly installed, and it puts only modest limitations on the size of data sets or models that can be analyzed.

09953_fm_ptg01_i-xxiv.indd 18 04/03/19 5:54 PM

SolverTable Add-in We also include SolverTable, a supplement to Excel’s built-in Solver for optimization.1 If you have ever had difficulty under- standing Solver’s sensitivity reports, you will appreciate SolverTable. It works like Excel’s data tables, except that for each input (or pair of inputs), the add-in runs Solver and reports the optimal output values. SolverTable is used extensively in the optimization chapters.

Windows versus Mac We have seen an increasing number of students using Macintosh laptops rather than Windows laptops. These students have two basic options when using our book. The first option is to use the latest version of Excel for Mac. Except for a few advanced tools such as Power Pivot (discussed in Chapter 4), the Mac version of Excel is very similar to the Windows version. However, the Palisade and SolverTable add-ins will not work with Excel for Mac. Therefore, the second option, the preferable option, is to use a Windows emulation program (Bootcamp and Parallels are good candidates), along with Office for Windows. Students at Indiana have used this second option for years and have had no problems.

Software Calculations by Chapter This section indicates how the various calculations are implemented in the book. Specifically, it indicates which calculations are performed with built-in Excel tools and which require Excel add-ins.

Important note: The Palisade add-ins used in several chapters do not work in Excel for Mac. This is the primary reason Albright developed his own DADM_Tools add-in, which works in Excel for Windows and Excel for Mac. This add-in is freely available at the author’s website (https://kelley.iu.edu/albrightbooks/free_downloads.htm), together with a Word document on how to use it. However, it is optional and is not used in the book.

Chapter 1 – Introduction to Business Analytics

• The section on basic spreadsheet modeling is implemented with built-in Excel functions.

Chapter 2 – Describing the Distribution of a Variable

• Everything is implemented with built-in Excel functions and charts.

° Summary measures are calculated with built-in functions AVERAGE, STDEV.S, etc. ° Histograms and box plots are created with the Excel chart types introduced in 2016. ° Time series graphs are created with Excel line charts.

• Palisade’s StatTools add-in can do all of this. It isn’t used in the chapter, but it is mentioned in a short appendix, and an Intro to StatTools video is available.

• Albright’s DADM_Tools add-in can do all of this except for time series graphs.

Chapter 3 – Finding Relationships among Variables

• Everything is implemented with built-in Excel functions and charts.

° Summary measures of numeric variables, broken down by categories of a categorical variable, are calculated with built-in functions AVERAGE, STDEV.S, etc. (They are embedded in array formulas with IF functions.)

° Side-by-side box plots are created with the Excel box plot chart type introduced in 2016. ° Scatterplots are created with Excel scatter charts. ° Correlations are calculated with Excel’s CORREL function. A combination of the CORREL and INDIRECT func-

tions is used to create tables of correlations.

• StatTools can do all of this. It isn’t used in the chapter, but it is mentioned in a short appendix.

• DADM_Tools can do all of this.

1 SolverTable is available on this textbook’s website and on Albright’s website, www.kelley.iu.edu/albrightbooks.

P R E F A C E x i x

09953_fm_ptg01_i-xxiv.indd 19 04/03/19 5:54 PM

Chapter 4 - Business Intelligence (BI) Tools for Data Analysis

• Queries are implemented with Excel’s Power Query tools (available only in Excel for Windows).

• Pivot table analysis is implemented with a combination of Excel’s Data Model and the Power Pivot add-in (available only in Excel for Windows).

• Tableau Public (a free program for Windows or Mac) is used for data visualization.

Chapter 5 – Probability and Probability Distributions

• All calculations are performed with built-in Excel functions.

Chapter 6 – Decision Making Under Uncertainty

• Decision trees are implemented with Palisade’s PrecisionTree add-in.

• DADM_Tools implements decision trees.

Chapter 7 – Sampling and Sampling Distributions

• All calculations are performed with built-in Excel functions.

Chapter 8 – Confidence Interval Estimation

• Everything is implemented with Excel functions, which are embedded in a Confidence Interval Template.xlsx file developed by Albright. This isn’t an add-in; it is a regular Excel file where the confidence interval formulas have already been created, and users only need to enter their data.

• StatTools can do all of this, but it isn’t used in the chapter.

• DADM_Tools doesn’t perform confidence interval calculations.

Chapter 9 – Hypothesis Testing

• Everything is implemented with Excel functions, which are embedded in Hypothesis Test Template.xlsx and Normality Tests Template.xlsx files developed by Albright. These aren’t add-ins; they are regular Excel files where the hypothesis test formulas have already been created, and users only need to enter their data.

• StatTools can do all of this, but it isn’t used in the chapter.

• DADM_Tools doesn’t perform hypothesis test calculations.

Chapters 10, 11 – Regression Analysis

• StatTools is used throughout to perform the regression calculations.

• Excel’s built-in regression functions (SLOPE, INTERCEPT, etc.) are illustrated for simple linear regression, but they are used sparingly.

• Excel’s Analysis ToolPak add-in is mentioned, but it isn’t used.

• DADM_Tools implements regression calculations.

Chapter 12 – Time Series Analysis and Forecasting

• Autocorrelations and the runs test for randomness are implemented with Excel functions, which are embedded in Auto- correlation Template.xlsx and Runs Test Template.xlsx files developed by Albright. These aren’t add-ins; they are reg- ular Excel files where the relevant formulas have already been created, and users only need to enter their data.

• Moving averages and exponential smoothing are implemented in StatTools.

• DADM_Tools implements moving averages and exponential smoothing calculations.

x x P R E F A C E

09953_fm_ptg01_i-xxiv.indd 20 04/03/19 5:54 PM

Chapters 13, 14 – Optimization

• The optimization is performed with Excel’s Solver add-in.

• Albright’s SolverTable add-in performs sensitivity analysis on optimal solutions. It works in Excel for Windows but not in Excel for Mac.

Chapters 15, 16 – Simulation

• Built-in Excel functions for generating random numbers from various distributions are illustrated, and the Excel-only way of running simulations with data tables is shown in an introductory example.

• Palisade’s @RISK add-in is used in the rest of the examples.

• DADM_Tools implements simulation.

Chapter 17 – Data Mining

• Logistic regression is implemented with StatTools.

• Neural networks are implemented with Palisade’s NeuralTools add-in.

• Other calculations are implemented with Excel functions.

Chapter 18 – Analysis of Variance and Experimental Design (online)

• Most of the chapter is implemented with StatTools.

• DADM_Tools doesn’t perform ANOVA calculations.

Chapter 19 – Statistical Process Control

• All control charts are implemented with StatTools.

• DADM_Tools doesn’t implement control charts.

Potential Course Structures Although we have used the book for our own required one-semester course, there is admittedly much more material than can be covered adequately in one semester. We have tried to make the book as modular as possible, allowing an instructor to cover, say, simulation before optimization or vice-versa, or to omit either of these topics. The one exception is statistics. Due to the natural progression of statistical topics, the basic topics in the early chapters should be covered before the more advanced topics (regression and time series analysis) in the later chapters. With this in mind, there are several possible ways to cover the topics.

• One-semester Required Course, with No Statistics Prerequisite (or where MBA students need a refresher for whatever statistics they learned previously): If data analysis is the primary focus of the course, then Chapters 2–3, 5, 7–11 should be covered. Depending on the time remaining, any of the topics in Chapters 4 and 17 (advanced data anal- ysis tools), Chapters 6 (decision making under uncertainty), 12 (time series analysis), 13–14 (optimization), or 15–16 (simulation) can be covered in practically any order.

• One-semester Required Course, with a Statistics Prerequisite: Assuming that students know the basic elements of statistics (up through hypothesis testing), the material in Chapters 2–3, 5, and 7–9 can be reviewed quickly, primarily to illustrate how Excel and add-ins can be used to do the number crunching. The instructor can then choose among any of the topics in Chapters 4, 6, 10–11, 12, 13–14, 15–16 (in practically any order), or 17 to fill the remainder of the course.

• Two-semester Required Sequence: Given the luxury of spreading the topics over two semesters, the entire book, or at least most of it, can be covered. The statistics topics in Chapters 2–3 and 7–9 should be covered in chronological order before other statistical topics (regression and time series analysis), but the remaining chapters can be covered in practi- cally any order.

P R E F A C E x x i

09953_fm_ptg01_i-xxiv.indd 21 04/03/19 5:54 PM

x x i i P R E F A C E

Instructor Supplements Textbook Website: www.cengage.com/decisionsciences/albright/ba/7e The companion website provides immediate access to an array of teaching resources—including data and solutions files for all of the Examples, Problems, and Cases in the book, Test Bank files, PowerPoint slides, and access to the DecisionTools® Suite by Palisade Corporation and the SolverTable add-in. Instructors who want to compare the problems in the previ- ous edition of this text to the problems in this edition can also download the file DADM 7e Problem Database.xlsx which details that correlation. You can easily download the instructor resources you need from the password-protected, instructor-only section of the site.

Test Bank Cengage Learning Testing Powered by Cognero® is a flexible, online system that allows you to import, edit, and manipulate content from the text’s Test Bank or elsewhere, including your own favorite test questions; create multiple test versions in an instant; and deliver tests from your LMS, your classroom, or wherever you want.

Student Supplements Textbook Website: www.cengage.com/decisionsciences/albright/ba/7e Every new student edition of this book comes with access to the Business Analytics: Data Analysis and Decision Making, 7e textbook website that links to the following files and tools:

• Excel files for the examples in the chapters (usually two versions of each— a template, or data-only version, and a finished version)

• Data files required for the Problems and Cases

• Excel Tutorial for Windows.xlsx, which contains a useful tutorial for getting up to speed in Excel (Excel Tutorial for the Mac.xlsx is also available)

• DecisionTools® Suite software by Palisade Corporation

• SolverTable add-in

09953_fm_ptg01_i-xxiv.indd 22 04/03/19 5:54 PM

P R E F A C E x x i i i

Acknowledgements We are also grateful to many of the professionals who worked behind the scenes to make this book a success: Aaron Arnsparger, Senior Product Manager; Brandon Foltz, Senior Learning Designer; Conor Allen, Content Manager; Project Manager, Anubhav Kaushal; and Marketing Manager, Chris Walz.

We also extend our sincere appreciation to the reviewers who provided feedback on the authors’ proposed changes that resulted in this seventh edition:

John Aloysius, Walton College of Business, University of Arkansas

Henry F. Ander, Arizona State University

Dr. Baabak Ashuri, School of Building Construction, Georgia Institute of Technology

James Behel, Harding University

Robert H. Burgess, Scheller College of Business, Georgia Institute of Technology

Paul Damien, McCombs School of Business, University of Texas in Austin

Parviz Ghandforoush, Virginia Tech

Betsy Greenberg, University of Texas

Anissa Harris, Harding University

Tim James, Arizona State University

Norman Johnson, C.T. Bauer College of Business, University of Houston

Shivraj Kanungo, The George Washington University

Miguel Lejeune, The George Washington University

José Lobo, Arizona State University

Stuart Low, Arizona State University

Lance Matheson, Virginia Tech

Patrick R. McMullen, Wake Forest University

Barbara A. Price, PhD, Georgia Southern University

Laura Wilson-Gentry, University of Baltimore

Toshiyuki Yuasa, University of Houston

S. Christian Albright

Wayne L. Winston

April 2019

09953_fm_ptg01_i-xxiv.indd 23 04/03/19 5:54 PM

09953_fm_ptg01_i-xxiv.indd 24 04/03/19 5:54 PM

CHAPTER 1 Introduction to Business Analytics

BUSINESS ANALYTICS PROVIDES INSIGHTS AND IMPROVES PERFORMANCE This book is about analyzing data and using quantitative modeling to help companies understand their business, make better decisions, and improve performance. We have been teaching the methods discussed in this book for decades, and companies have been using these methods for decades to improve performance and save millions of dollars. There- fore, we were a bit surprised when a brand new term, Busi- ness Analytics (BA), became hugely popular several years ago. All of a sudden, BA promised to be the road to success. By using quantitative BA methods—data analysis, optimiza-

tion, simulation, prediction, and others—companies could drastically improve business performance. Haven’t those of us in our field been doing this for years? What is different about BA that has made it so popular, both in the academic world and even more so in the business world?

The truth is that BA does use the same quantitative methods that have been used for years, the same methods you will learn in this book. BA has not all of a sudden invented brand new quantitative methods to eclipse traditional methods. The main difference is that BA uses big data to solve business problems and provide insights. Companies now have access to huge sources of data, and the technology is now available to use huge data sets for quantitative analysis, predictive modeling, optimization, and simulation. In short, the same quantitative methods that have been used for years can now be even more effective by utilizing big data and the corresponding technology.

For a quick introduction to BA, you should visit the BA Wikipedia site (search the Web for “business analytics”). Among other things, it lists areas where BA plays a prom- inent role, including the following: retail sales analytics; financial services analytics; risk and credit analytics; marketing analytics; pricing analytics; supply chain analytics; and transportation analytics. If you glance through the examples and problems in this book, you will see that most of them come from these same areas. Again, the difference is that we use relatively small data sets to get you started—we do not want to overwhelm you with gigabytes of data—whereas real applications of BA use huge data sets to advantage.

A more extensive discussion of BA can be found in the Fall 2011 research report, Analytics: The Widening Divide, published in the MIT Sloan Management Review in collaboration with IBM, a key developer of BA software (search the Web for the arti- cle’s title). This 22-page article discusses what BA is and provides several case studies. In addition, it lists three key competencies people need to compete successfully in the BA world—and hopefully you will be one of these people.

• Competency 1: Information management skills to manage the data. This competency involves expertise in a variety of techniques for managing data. Given the key role of data in BA methods, data quality is extremely important. With data coming from a number of disparate sources, both internal and external to an organization, achieving data quality is no small feat.

S to

c kL

ite /S

h u tt e rs

to c k.

c o m

09953_ch01_ptg01_001-036.indd 1 04/03/19 10:51 PM

2 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

• Competency 2: Analytics skills and tools to understand the data. We were not surprised, but rather very happy, to see this competency listed among the requirements because these skills are exactly the skills we cover throughout this book—optimization with advanced quantitative algorithms, simulation, and others.

• Competency 3: Data-oriented culture to act on the data. This refers to the culture within the organization. Everyone involved, especially top management, must believe strongly in fact-based decisions arrived at using analytical methods.

The article argues persuasively that the companies that have these competencies and have embraced BA have a distinct competitive advantage over companies that are just start- ing to use BA methods or are not using them at all. This explains the title of the article. The gap between companies that embrace BA and those that do not will only widen in the future.

This field of BA and, more specifically, data analysis is progressing very quickly, not only in academic areas but also in many applied areas. By analyzing big data sets from various sources, people are learning more and more about the way the world works. One recent book, Everybody Lies: Big Data New Data, and What the Internet Can Tell Us About Who We Really Are, by Stephens-Davidowitz (2017), is especially interesting. The title of the book suggests that people lie to themselves, friends, Face- book, and surveys, but they don’t lie on Google searches. The author, a social scientist, has focused on the data from Google searches to study wide-ranging topics, including economics, finance, education, sports, gambling, racism, and sex. Besides his insightful and often unexpected findings, the author presents several general conclusions about analysis with big data:

• Not surprisingly, insights are more likely to be obtained when analyzing data in novel ways, such as analyzing the organ sizes of race horses instead of their pedigrees. (A data analyst used data on race horse heart size to predict that American Pharaoh would be great. The horse went on to win the Triple Crown in 2015, the first horse to do so since 1978.) On the other hand, if all available data have been analyzed in every conceivable way, as has been done with stock market data, even analysis with big data is unlikely to produce breakthroughs.

• Huge data sets allow analysts to zoom in on smaller geographical areas or smaller subsets of a population for keener insights. For example, when analyzing why some areas of the U.S. produce more people who warrant an entry on Wikipedia, you might find that richer and more educated states have more success. However, with a data set of more than 150,000 Americans of Wikipedia fame, the author was able to pinpoint that a disproportionate number were born in college towns. Traditional small data sets don’t allow drilling down to this level of detail.

• Companies are increasingly able to run controlled experiments, usually on the Web, to learn the causes of behavior. The new term for these experiments is A/B testing. The purpose of a test is to determine whether a person prefers A or B. For example, Google routinely runs A/B tests, showing one randomly selected group of customers one thing and a second randomly selected group another. The difference could be as slight as the presence or absence of an arrow link on a Web page. Google gets immediate results from such tests by studying the customers’ clicking behavior. The company can then use the results of one test to lead to another follow-up test. A/B testing is evidently happening all the time, whether or not we’re aware of it.

• A potential drawback of analyzing big data is dimensionality, where there are so many potential explanatory variables that one will almost surely look like a winner—even though it isn’t. The author illustrates this by imagining daily coin flips of 1000 coins and tracking which of them comes up heads on days when the stock market is up. One of these coins, just by chance, is likely to correlate highly with the stock market, but this coin is hardly a useful predictor of future stock market movement.

09953_ch01_ptg01_001-036.indd 2 04/03/19 10:51 PM

1-1 Introduction 3

The analysis of big data will not solve all the world’s problems, and there are ethical issues about studying people’s behavior—and acting on the findings—from Web-based data such as Google searches. However, the potential for new and important insights into human behavior is enormous.

1-1 Introduction We are living in the age of technology. This has two important implications for every- one entering the business world. First, technology has made it possible to collect huge amounts of data. Technology companies like Google and Amazon capture click data from websites, retailers collect point-of-sale data on products and customers every time a trans- action occurs; credit agencies collect data on people who have or would like to obtain credit; investment companies have a limitless supply of data on the historical patterns of stocks, bonds, and other securities; and government agencies have data on economic trends, the environment, social welfare, consumer product safety, and virtually everything else imaginable. It has become relatively easy to collect the data. As a result, data are plen- tiful. However, as many organizations have discovered, it is a challenge to make sense of the data they have collected.

A second important implication of technology is that it has given many more people the power and responsibility to analyze data and make decisions on the basis of quantitative analysis. People entering the business world can no longer pass all the quantitative analy- sis to the “quant jocks,” the technical specialists who have traditionally done the number crunching. The vast majority of employees now have a computer at their disposal, access to relevant data, and training in easy-to-use software, particularly spreadsheet and database software. For these employees, quantitative methods are no longer forgotten topics they once learned in college. Quantitative analysis is now an essential part of their daily jobs.

A large amount of data already exists, and it will only increase in the future. Many companies already complain of swimming in a sea of data. However, enlightened com- panies are seeing this expansion as a source of competitive advantage. In fact, one of the hottest topics in today’s business world is business analytics, also called data analytics. These terms have been created to encompass the types of analysis discussed in this book, so they aren’t really new; we have been teaching them for years. The new aspect of busi- ness analytics is that it typically implies the analysis of very large data sets, the kind that companies currently encounter. For this reason, the term big data has also become popular. By using quantitative methods to uncover the information from the data and then act on this information—again guided by quantitative analysis—companies are able to gain advantages over their less enlightened competitors. Here are several pertinent examples.

• Direct marketers analyze enormous customer databases to see which customers are likely to respond to various products and types of promotions. Marketers can then target different classes of customers in different ways to maximize profits—and give their customers what they want.

• Hotels and airlines also analyze enormous customer databases to see what their customers want and are willing to pay for. By doing this, they have been able to devise very clever pricing strategies, where different customers pay different prices for the same accommodations. For example, a business traveler typically makes a plane reservation closer to the time of travel than a vacationer. The airlines know this. Therefore, they reserve seats for these business travelers and charge them a higher price for the same seats. The airlines profit from clever pricing strategies, and the customers are happy.

• Financial planning services have a virtually unlimited supply of data about security prices, and they have customers with widely differing preferences for various types of investments. Trying to find a match of investments to customers is a very

09953_ch01_ptg01_001-036.indd 3 04/03/19 10:51 PM

4 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

challenging problem. However, customers can easily take their business elsewhere if good decisions are not made on their behalf. Therefore, financial planners are under extreme competitive pressure to analyze masses of data so that they can make informed decisions for their customers.1

• We all know about the pressures U.S. manufacturing companies have faced from foreign competition in the past few decades. The automobile companies, for example, have had to change the way they produce and market automobiles to stay in business. They have had to improve quality and cut costs by orders of magnitude. Although the struggle continues, much of the success they have had can be attributed to data analysis and wise decision making. Starting on the shop floor and moving up through the organization, these companies now measure almost everything, analyze these measurements, and then act on the results of their analysis.

We talk about companies analyzing data and making decisions. However, companies don’t really do this; people do it. And who will these people be in the future? They will be you! We know from experience that students in all areas of business, at both the undergraduate and graduate level, will be required to analyze large complex data sets, run regression anal- yses, make quantitative forecasts, create optimization models, and run simulations. You are the person who will be analyzing data and making important decisions to help your com- pany gain a competitive advantage. And if you are not willing or able to do so, there will be plenty of other technically trained people who will be more than happy to do it.

The goal of this book is to teach you how to use a variety of quantitative methods to analyze data and make decisions in a very hands-on way. We discuss a number of quan- titative methods and illustrate their use in a large variety of realistic business situations. As you will see, this book includes many examples from finance, marketing, operations, accounting, and other areas of business. To analyze these examples, we take advantage of the Microsoft Excel® spreadsheet software, together with several powerful Excel add-ins. In each example we provide step-by-step details of the method and its implementation in Excel.

This is not a “theory” book. It is also not a book where you can lean comfortably back in your chair and read about how other people use quantitative methods. It is a “get your hands dirty” book, where you will learn best by actively following the examples through- out the book on your own computer. By the time you have finished, you will have acquired some very useful skills for today’s business world.

1-2 Overview of the Book This section provides an overview of the methods covered in this book and the software that is used to implement them. Then the rest of the chapter illustrates how some of Excel’s basic tools can be used to solve quantitative problems.

1-2a The Methods This book is rather unique in that it combines topics from two separate fields: statistics and management science. Statistics is the area of data analysis, whereas management sci- ence is the area of model building, optimization, and decision making. In the academic arena these two fields have traditionally been separated, sometimes widely. Indeed, they are often housed in separate academic departments. However, both are useful in accom- plishing what the title of this book promises: data analysis and decision making.

Therefore, we do not distinguish between the statistics and the management science parts of this book. Instead, we view the entire book as a collection of useful quantita- tive methods for analyzing data and helping to make business decisions. In addition, our

1 For a great overview of how quantitative techniques have been used in the financial world, read the book The Quants, by Scott Patterson. It describes how quantitative models made millions for a lot of bright young analysts, but it also describes the dangers of relying totally on quantitative models, at least in the complex world of global finance.

09953_ch01_ptg01_001-036.indd 4 04/03/19 10:51 PM

1-2 Overview of the Book 5

choice of software helps to integrate the various topics. By using a single package, Excel, together with several Excel add-ins, you will see that the methods of statistics and man- agement science are similar in many important respects.

Three important themes run through this book. Two of them are in the title: data anal- ysis and decision making. The third is dealing with uncertainty.2 Each of these themes has subthemes. Data analysis includes data description, data visualization, data inference, and the search for relationships in data. Decision making includes optimization techniques for problems with no uncertainty, decision analysis for problems with uncertainty, and structured sensitivity analysis. Dealing with uncertainty includes measuring uncertainty and modeling uncertainty explicitly. There are obvious overlaps between these themes and subthemes. When you make inferences from data and search for relationships in data, you must deal with uncertainty. When you use decision trees to help make decisions, you must deal with uncertainty. When you use simulation models to help make decisions, you must deal with uncertainty, and then you often make inferences from the simulation results.

Figure 1.1 shows where these themes and subthemes are discussed in the book. The next few paragraphs discuss the book’s contents in more detail.

2 The fact that the uncertainty theme did not find its way into the title of this book does not detract from its importance. We just wanted to keep the title reasonably short.

Figure 1.1 Themes and Subthemes

We begin in Chapters 2, 3, and 4 by illustrating a number of ways to summarize the information in data sets. These include graphical and tabular summaries, as well as numeric summary measures such as means, medians, and standard deviations. As stated earlier, organizations are now able to collect huge amounts of raw data, but what does it all mean? Although there are very sophisticated methods for analyzing data, some of which are covered in later chapters, the “simple” methods in Chapters 2, 3, and 4 are crucial for obtaining an initial understanding of the data. Fortunately, Excel and available add-ins now make this quite easy. For example, Excel’s pivot table tool for “slicing and dicing” data is an analyst’s dream come true. You will be amazed at the insights you can gain from pivot tables—with very little effort.

Uncertainty is a key aspect of most business problems. To deal with uncertainty, you need a basic understanding of probability. We discuss the key concepts in Chapter 5. This chapter covers basic rules of probability and then discusses the extremely important con- cept of probability distributions, with emphasis on two of the most important probability distributions, the normal and binomial distributions.

Themes Subthemes Chapters Where Emphasized

2−4, 10, 12, 17

7−9, 11, 18−19

3, 10−12, 17−18

6, 13−16

5−12, 15−16, 18−19

5−6, 10−12, 15−16, 18−19

13, 14

Data Analysis

Description

Inference

Relationships

Optimization

Decision Analysis with Uncertainty

Sensitivity Analysis

Measuring

Modeling

Decision Making

Uncertainty

09953_ch01_ptg01_001-036.indd 5 04/03/19 10:51 PM

6 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

In Chapter 6 we apply probability to decision making under uncertainty. These types of problems—faced by all companies on a continual basis—are characterized by the need to make a decision now, even though important information, such as demand for a product or returns from investments, will not be known until later. The methods in Chapter 6 pro- vide a rational basis for making such decisions.

In Chapters 7, 8, and 9 we discuss sampling and statistical inference. Here the basic problem is to estimate one or more characteristics of a population. If it is too expensive or time-consuming to learn about the entire population—and it usually is—it is instead com- mon to select a random sample from the population and then use the information in the sample to infer the characteristics of the population.

In Chapters 10 and 11 we discuss the extremely important topic of regression analy- sis, which is used to study relationships between variables. The power of regression anal- ysis is its generality. Every part of a business has variables that are related to one another, and regression can often be used to estimate relationships between these variables.

From regression, we move to time series analysis and forecasting in Chapter 12. This topic is particularly important for providing inputs into business decision problems. For example, manufacturing companies must forecast demand for their products to make sen- sible decisions about order quantities from their suppliers. Similarly, fast-food restaurants must forecast customer arrivals, sometimes down to the level of 15-minute intervals, so that they can staff their restaurants appropriately. Chapter 12 illustrates some of most fre- quently used forecasting methods.

Chapters 13 and 14 are devoted to spreadsheet optimization. We assume a company must make several decisions, and there are constraints that limit the possible decisions. The job of the decision maker is to choose the decisions such that all the constraints are satisfied and an objective, such as total profit or total cost, is optimized. The solution pro- cess consists of two steps. The first step is to build a spreadsheet model that relates the decision variables to other relevant quantities by means of logical formulas. The second step is then to find the optimal solution. Fortunately, Excel contains a Solver add-in that performs the optimization. All you need to do is specify the objective, the decision vari- ables, and the constraints; Solver then uses powerful algorithms to find the optimal solu- tion. As with regression, the power of this approach is its generality.

Chapters 15 and 16 illustrate a number of computer simulation models. As mentioned earlier, most business problems have some degree of uncertainty. The demand for a prod- uct is unknown, future interest rates are unknown, the delivery lead time from a supplier is unknown, and so on. Simulation allows you to build this uncertainty explicitly into spread- sheet models. Some cells in the model contain random values with given probability dis- tributions. Every time the spreadsheet recalculates, these random values change, which causes “bottom-line” output cells to change as well. The trick then is to force the spread- sheet to recalculate many times and keep track of interesting outputs. In this way you can see an entire distribution of output values that might occur, not just a single best guess.

Chapter 17 then returns to data analysis. It provides an introduction to data mining, a topic of increasing importance in today’s data-driven world. Data mining is all about exploring data sets, especially large data sets, for relationships and patterns that can help companies gain a competitive advantage. It employs a number of relatively new technolo- gies to implement various algorithms, several of which are discussed in this chapter.

Finally, there are two online chapters, 18 and 19, that complement topics included in the book itself. Chapter 18 discusses analysis of variance (ANOVA) and experimen- tal design. Chapter 19 discusses quality control and statistical process control. These two online chapters follow the same structure as the chapters in the book, complete with many examples and problems.

1-2b The Software The methods in this book can be used to analyze a wide variety of business problems. However, they are not of much practical use unless you have the software to implement them. Very few business problems are small enough to be solved with pencil and paper. They require powerful software.

09953_ch01_ptg01_001-036.indd 6 04/03/19 10:51 PM

1-2 Overview of the Book 7

The software available with this book, together with Microsoft Excel, provides you with a powerful combination. This software is being used—and will continue to be used— by leading companies all over the world to analyze large, complex problems. We firmly believe that the experience you obtain with this software, through working the examples and problems in the book, will give you a key competitive advantage in the business world.

It all begins with Excel. Almost all the quantitative methods discussed in the book are implemented in Excel. We cannot forecast the state of computer software in the long-term future, but Excel is currently the most heavily used spreadsheet package on the market, and there is every reason to believe that this state will persist for many years. Most compa- nies use Excel, most employees and most students have been trained in Excel, and Excel is a very powerful, flexible, and easy-to-use package.

It helps to understand Microsoft’s versions of Excel. Until recently, we referred to Excel 2007, Excel 2010, and so on. Every few years, Microsoft released a new version of Office, and hence Excel. The latest was Excel 2016, and the next is going to be Excel 2019, which might be out by the time you read this. Each of these versions is basically fixed. For example, if you had Excel 2013, you had to wait for Excel 2016 to get the newest features. However, within the past few years, Microsoft has offered a subscription service, Office 365, which allows you to install free updates about once per month. Will Microsoft eventually discontinue the fixed versions and push everyone to the subscription service? We don’t know, but it would be nice. Then we could assume that everyone had the same version of Excel, with all the latest features. Actually, this isn’t quite true. There are different levels of Microsoft Office 365, some less expensive but with fewer features. You can find details on Microsoft’s website. This book assumes you have the features in Excel 2016 or the ProPlus version of Office 365. You can see your version by opening Excel and selecting Account from the File menu.

We also realize that many of you are using Macs. There is indeed a version of Office 365 for the Mac, but its version of Excel doesn’t yet have all the features of Excel for Windows. For example, we discuss two of Microsoft’s “power” tools, Power Query and Power Pivot, in Chapter 4. At least as of this writing, these tools are not available in Excel for Mac. On the other hand, some features, including pivot charts, histograms, and box plots, were not in Office 365 for Mac until recently, when they suddenly appeared. So Microsoft is evidently catching up. In addition, some third-party add-ins for Excel, notably the Palisade DecisionTools Suite used in some chapters, are not available for Macs and probably never will be. If you really need these features and want to continue using your Mac, your best option is to install Windows emulation software such as Boot Camp or Parallels. Our students at Indiana have been doing this successfully for years.

Built-in Excel Features Virtually everyone in the business world knows the basic features of Excel, but relatively few know some of its more powerful features. In short, relatively few people are the “power users” we expect you to become by working through this book. To get you started, the files Excel Tutorial for Windows.xlsx and Excel Tutorial for the Mac.xlsx explain some of the “intermediate” features of Excel—features that we expect you to be able to use. (See the Preface for instructions on how to access the resources that accompany this textbook.) These include the SUMPRODUCT, VLOOKUP, IF, NPV, and COUNTIF, functions. They also include range names, data tables, Paste Special, Goal Seek, and many others. Finally, although we assume you can perform routine spreadsheet tasks such as copying and pasting, the tutorial provides many tips to help you perform these tasks more efficiently.3

In addition to this tutorial, the last half of this chapter presents several examples of modeling quantitative problems in Excel. These examples provide a head start in using Excel tools you will use in the rest of the book. Then later chapters will introduce other useful Excel tools as they are needed.

3 Albright and several colleagues have created a more robust commercial version of this tutorial called excelNow!. The Excel Tutorial files explain how you can upgrade to this commercial version.

09953_ch01_ptg01_001-036.indd 7 04/03/19 10:51 PM

8 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

Analysis ToolPak All versions of Excel, extending back at least two decades, have included an add-in called Analysis ToolPak. It has several tools for data analysis, including correlation, regression, and inference. Unfortunately, Microsoft hasn’t updated Analysis ToolPak for years, so it is not as good as it could be. We will mention Analysis ToolPak a few times throughout the book, but we will use other Excel tools whenever they are available.

Solver Add-in Chapters 13 and 14 make heavy use of Excel’s Solver add-in. This add-in, developed by Frontline Systems®, not Microsoft, uses powerful algorithms to perform spreadsheet opti- mization. Before this type of spreadsheet optimization add-in was available, specialized (nonspreadsheet) software was required to solve optimization problems. Now you can do it all within the familiar Excel environment.

SolverTable Add-in An important theme throughout this book is sensitivity analysis: How do outputs change when inputs change? Typically these changes are made in spreadsheets with a data table, a built-in Excel tool. However, data tables don’t work in optimization models, where the goal is to see how the optimal solution changes when certain inputs change. Therefore, we include an Excel add-in called SolverTable, which works almost exactly like Excel’s data tables. (This add-in was developed for Excel for Windows by Albright. Unfortunately, it doesn’t work for Excel for Mac.) Chapters 13 and 14 illustrate the use of SolverTable.

Palisade DecisionTools Suite In addition to SolverTable and built-in Excel add-ins, an educational version of Palisade Corporation’s powerful DecisionTools® Suite is available. All programs in this suite are Excel add-ins, so the learning curve isn’t very steep. There are seven separate add-ins in this suite: @RISK, BigPicture, StatTools, PrecisionTree, NeuralTools, TopRank, and Evolver. The add-ins we will use most often are StatTools (for data analysis), Precision- Tree (for decision trees), and @RISK (for simulation). These add-ins will be discussed in some detail in the chapters where they are used.

DADM_Tools Add-In We realize that some of you prefer not to use the Palisade software because it might not be available in companies where your students are eventually employed. Nevertheless, some of the methods discussed in the book, particularly decision trees and simulation, are difficult to implement with Excel tools only. Therefore, Albright recently developed an add-in called DADM_Tools that implements decision trees and simulation, as well as forecasting and several basic data analysis tools. This add-in is freely available from the author’s website at https://kelley.iu.edu/albrightbooks/free_downloads.htm, and students can continue to use it after the course is over, even in their eventual jobs, for free. You can decide whether you want to use the Palisade software, the DADM_Tools add-in, or neither.

1-3 Introduction to Spreadsheet Modeling4 A common theme in this book is spreadsheet modeling, where the essential elements of a business problem are entered and related in an Excel spreadsheet for further analysis. This section provides an introduction to spreadsheet modeling with some relatively simple models. Together with the Excel tutorial mentioned previously, the goal of this section is to get you “up to speed” in using Excel effectively for the rest of the book.

4 If you (or your students) are already proficient in basic Excel tools, you can skip this section, which is new to this edition of the book.

09953_ch01_ptg01_001-036.indd 8 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 9

1-3a Basic Spreadsheet Modeling: Concepts and Best Practices

Most spreadsheet models involve inputs, decision variables, and outputs. The inputs have given fixed values, at least for the purposes of the model. The decision variables are those a decision maker controls. The outputs are the ultimate values of interest; they are deter- mined by the inputs and the decision variables.

Spreadsheet modeling is the process of entering the inputs and decision variables into a spreadsheet and then relating them appropriately, by means of formulas, to obtain the outputs. After you have done this, you can then proceed in several directions. You might want to perform a sensitivity analysis to see how one or more outputs change as selected inputs or decision variables change. You might want to find the values of the decision variable(s) that minimize or maximize a particular output, possibly subject to certain con- straints. You might also want to create charts that show graphically how certain parameters of the model are related.

Getting all the spreadsheet logic correct and producing useful results is a big part of the battle. However, it is also important to use good spreadsheet modeling practices. You probably won’t be developing spreadsheet models for your sole use; instead, you will be sharing them with colleagues or even a boss (or an instructor). The point is that other peo- ple will be reading and trying to make sense out of your spreadsheet models. Therefore, you should construct your spreadsheet models with readability in mind. Features that can improve readability include the following:5

• A clear, logical layout to the overall model • Separation of different parts of a model, possibly across multiple worksheets • Clear headings for different sections of the model and for all inputs, decision variables,

and outputs • Use of range names • Use of boldface, italics, larger font size, coloring, indentation, and other formatting

features • Use of cell comments • Use of text boxes for assumptions and explanations

The following example illustrates the process of building a spreadsheet model accord- ing to these guidelines. We build this model in stages. In the first stage, we build a model that is correct, but not very readable. At each subsequent stage, we modify the model to enhance its readability.

5 For further guidelines that attempt to make spreadsheet models more flexible and less prone to errors, see the article by LeBlanc et al. (2018).

EXAMPLE

1.1 ORDERING NCAA T-SHIRTS It is March, and the annual NCAA Basketball Tournament is down to the final four teams. Randy Kitchell is a T-shirt vendor who plans to order T-shirts with the names of the final four teams from a manufacturer and then sell them to the fans. The fixed cost of any order is $750, the variable cost per T-shirt to Randy is $8, and Randy’s selling price is $18. However, this price will be charged only until a week after the tournament. After that time, Randy figures that interest in the T-shirts will be low, so he plans to sell all remaining T-shirts, if any, at $6 each. His best guess is that demand for the T-shirts during the full-price period will be 1500. He is thinking about ordering 1450 T-shirts, but he wants to build a spreadsheet model that will let him experiment with the uncertain demand and his order quantity. How should he proceed?

Objective To build a spreadsheet model in a series of stages, with all stages being correct but each stage being more readable and flexible than the previous stages.

09953_ch01_ptg01_001-036.indd 9 04/03/19 10:51 PM

Solution The logic behind the model is fairly simple, but the model is built for generality. Specifically, the formulas allow for the order quantity to be less than, equal to, or greater than demand. If demand is greater than the order quantity, Randy will sell all the T-shirts ordered for $18 each. If demand is less than the order quantity, Randy will sell as many T-shirts as are demanded at the $18 price and all leftovers at the $6 price. You can implement this logic in Excel with an IF function.

A first attempt at a spreadsheet model appears in Figure 1.2. (See the file TShirt Sales Finished.xlsx, where each stage appears on a separate worksheet.) You enter a possible demand in cell B3, a possible order quantity in cell B4, and then calcu- late the profit in cell B5 with the formula

5−750−8*B41IF(B3>B4,18*B4,18*B316*(B4−B3))

This formula subtracts the fixed and variable costs and then adds the revenue according to the logic just described.

This is exactly the same formula as before, but it is now more flexible. If an input changes, the profit recalculates automati- cally. Most important, the inputs are no longer buried in the formula.

Still, the profit formula is not very readable as it stands. You can make it more readable by using range names. The mechanics of range names are covered in detail later in this section. For now, the results of using range names for cells

This model in Figure 1.2 is entirely correct, but it isn’t very readable or flexible because it breaks a rule that you should never break: It hard codes input values into the profit formula. A spreadsheet model should never include input numbers in formulas. Instead, it should store input values in sepa- rate cells and then use cell references to these inputs in its formulas. A remedy appears in Figure 1.3, where the inputs have been entered in the range B3:B6, and the profit formula in cell B10 has been changed to

5−B3−B4*B91IF(B8>B9,B5*B9,B5*B81B6*(B9−B8))

Never hard code numbers into Excel formulas. Use cell references instead.

1 0 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

1 2 3 4 5

A B NCAA t-shirt sales

Demand Order Profit

1500 1450

13750

Figure 1.2 Base Model

1 2 3 4 5 6 7 8 9

A B NCAA t-shirt sales

Fixed order cost Variable cost Selling price Discount price

Demand Order Profit

$750 $8

$18 $6

1500 1450

$13,750

Figure 1.3 Model with Input Cells

Excel’s IF function has the syntax =IF(condition, result_if_True,result_if_False). The condition is any expression that is either true or false. The two expressions result_if_True and result_if_False can be any expressions you would enter in a cell: numbers, text, or other Excel functions (including other IF functions). If either expression is text, it must be enclosed in double quotes, such as 5IF(Score.590,"A","B"). Also, the condition can be complex combinations of conditions, using the keywords AND or OR. Then the syntax is, for example, 5IF(AND(Score1,60,Score2,60),"Fail","Pass").

Excel Function

09953_ch01_ptg01_001-036.indd 10 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 1 1

B3 through B6, B8, and B9 are shown in Figure 1.4. This model looks exactly like the previous model, but the formula in cell B10 is now

5−Fixed_order_cost−Variable_cost*Order1IF(Demand>Order, Selling_price*Order,Selling_price*Demand 1Discount_Price*(Order−Demand))

This formula is admittedly more long-winded, but it is easier to read.

Next, Randy might like to have profit broken down into various costs and revenues as shown in Figure 1.5, rather than one single profit cell. The formulas in cells B12, B13, B15, and B16 are straightforward, so they are not repeated here. You can then accumulate these to get profit in cell B17 with the formula

5−(B121B13)1(B151B16)

Of course, range names could be used for these intermediate output cells, but this is probably more work than it’s worth. You should always use discretion when deciding how many range names to use.

If you see the model in Figure 1.5, how do you know which cells contain inputs or decision variables or outputs? Labels and/or color coding can help to distinguish these types. A blue/red/gray color-coding style has been applied in Figure 1.6, along with descriptive labels in boldface. The blue cells at the top are input cells, the red cell in the middle is a decision vari- able, and the gray cell at the bottom is the key output. There is nothing sacred about this particular convention. Feel free to adopt your own convention and style, but use it consistently.

1 2 3 4 5 6 7 8 9

A B C D E F NCAA t-shirt sales

Fixed order cost Range names used Variable cost Selling price Discount price

Order Demand Order Profit

$750 $8

$18 $6

1500 1450

$13,750

Demand Discount_price Fixed_order_cost

Selling_price Variable_cost

='Model 3'!$B$8 ='Model 3'!$B$6 ='Model 3'!$B$3 ='Model 3'!$B$9 ='Model 3'!$B$5 ='Model 3'!$B$4

Figure 1.4 Model with Range Names in Profit Formula

Figure 1.5 Model with Intermediate Outputs

Spreadsheet Layout and Documentation

If you want your spreadsheets to be used (and you want your value in your company to rise), think carefully about your spreadsheet layout and then document your work carefully. For layout, consider whether certain data are best oriented in rows or columns, whether your work is better placed in a single sheet or in multiple sheets, and so on. For documentation, use descriptive labels and headings, color coding, cell comments, and text boxes to make your spreadsheets more readable. It takes time and careful planning to design and then document your spreadsheet models, but the time is well spent. And if you come back in a few days to a spreadsheet model you developed and you can’t make sense of it, don’t be afraid to redesign your work completely—from the ground up.

Fundamental Insight

09953_ch01_ptg01_001-036.indd 11 04/03/19 10:51 PM

1-3b Cost Projections In the following example, a company wants to project its costs of producing products, given that material and labor costs are likely to increase through time. We build a simple model and then use Excel’s charting capabilities to obtain a graphical image of projected costs.

The model in Figure 1.6 is still not the last word on this example. As shown in later examples, you can create data tables to see how sensitive profit is to the inputs, the demand, and the order quantity. You can also create charts to show results graph- ically. But this is enough for now. You can see that the model in Figure 1.6 is now much more readable and flexible than the original model in Figure 1.2.

1 2 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22

A B C D E F NCAA t-shirt sales

Input egnaRselbairav names used Fixed order cost $750 Demand ='Model 5'!$B$10 Variable cost $8 Discount_price ='Model 5'!$B$7 Selling price $18 Fixed_order_cost ='Model 5'!$B$4 Discount price $6 Order ='Model 5'!$B$13

Selling_price ='Model 5'!$B$6 Uncertain variable Variable_cost ='Model 5'!$B$5

0051dnameD

Decision variable 0541redrO

Output variables Costs

Fixed cost $750 Variable costs $11,600

Revenues Full-price shirts $26,100 Discount-price shirts $0

$13,750tiforP

Figure 1.6 Model with Category Labels and Color Coding

EXAMPLE

1.2 PROJECTING THE COSTS OF BOOKSHELVES AT WOODWORKS The Woodworks Company produces a variety of custom-designed wood furniture for its customers. One favorite item is a bookshelf, made from either cherry or oak. The company knows that wood prices and labor costs are likely to increase in the future. Table 1.1 shows the number of board-feet and labor hours required for a bookshelf, the current costs per board-foot and labor hour, and the anticipated annual increases in these costs. (The top row indicates that either type of bookshelf requires 30 board-feet of wood and 16 hours of labor.) Build a spreadsheet model that enables the company to experiment with the growth rates in wood and labor costs so that a manager can see, both numerically and graphically, how the costs of the book- shelves increase in the next few years.

resource Cherry Oak Labor

Required per bookshelf 30 30 16

Current unit cost $5.50 $4.30 $18.50

Anticipated annual cost increase

2.4% 1.7% 1.5%

Table 1.1 Input Data for Manufacturing a Bookshelf

Objective To learn good spreadsheet practices, to create copyable formulas with the careful use of relative and absolute addresses, and to create line charts from multiple series of data.

09953_ch01_ptg01_001-036.indd 12 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 1 3

Solution The completed spreadsheet model appears in Figure 1.7 and in the file Bookshelf Costs Finished.xlsx. You can develop it with the following steps.

1. Inputs. You should usually enter the inputs for a model in the upper-left corner of a work- sheet as you can see in the shaded ranges in Figure 1.7. We have used our standard conven- tion of coloring inputs—the numbers from the statement of the problem—blue. You can develop your own convention, but the input cells should be distinguished in some way. Note that the inputs are grouped logically and are explained with appropriate labels. You should always document your spreadsheet model with descriptive labels. Also, note that by entering inputs explicitly in input cells, you can refer to them later in Excel formulas.

Always enter input values in input cells and then refer to them in Excel formulas. Do not bury input values in formulas.

Figure 1.7 Bookshelf Cost Model

Relative and Absolute Addresses in Formulas

Relative and absolute addresses are used in Excel formulas to facilitate copying. A dollar sign next to a column or row address indicates that the address is absolute and will not change when copying. The lack of a dollar sign indi- cates that the address is relative and will change when copying. After you select a cell in a formula, you can press the F4 key repeatedly to cycle through the relative/absolute possibilities: =B4 (both column and row relative); =$B$4 (both column and row absolute); =B$4 (column relative, row absolute); and =$B4 (column absolute, row relative).

Excel Tip

Always try to organize your spreadsheet model so that you can copy formulas across multiple cells.

2. Design output table. Plan ahead for how you want to structure your outputs. We created a table where there is a row for every year in the future (year 0 corresponds to the current year), there are three columns for projected unit costs (columns B–D), and there are two columns for projected total bookshelf costs (columns E and F). The headings reflect this design. This isn’t the only possible design, but it works well. The important point is that you should have some logical design in mind before you dive in.

3. Projected unit costs of wood. The dollar values in the range B19:F25 are calculated from Excel formulas. Although the logic in this example is straightforward, it is still important to have a strategy in mind before you enter formulas. In particular, you should always try to design your spreadsheet so that you can enter a single formula and then copy it. This saves work and avoids errors. For the costs per board-foot in columns B and C, enter the formula

=B9

in cell B19 and copy it to cell C19. Then enter the general formula

5B19*(11B$10)

in cell B20 and copy it to the range B20:C25. We assume you know the rules for absolute and relative addresses (dollar sign for absolute, no dollar sign for relative), but it takes some planning to use these so that copying is possible. Make sure you understand why we made row 10 absolute but column B relative.

Typing dollar signs in formulas for absolute references is inefficient. Press the F4 key instead.

Press the Fn key and the F4 key (together) on Mac keyboards.

09953_ch01_ptg01_001-036.indd 13 04/03/19 10:51 PM

4 Projected unit labor costs. To calculate projected hourly labor costs, enter the formula

=B13

in cell D19. Then enter the formula

5D19*(11B$14)

in cell D20 and copy it down column D.

5 Projected bookshelf costs. Each bookshelf cost is the sum of its wood and labor costs. By a careful use of absolute and relative addresses, you can enter a single formula for these costs—for all years and for both types of wood. To do this, enter the formula

5B$5*B191B$6*$D19

in cell E19 and copy it to the range E19:F25. The idea here is that the units of wood and labor per bookshelf are always in rows 5 and 6, and the projected unit labor cost is always in column D, but all other references are relative to allow copying.

6 Chart. A chart is a valuable addition to any table of data, especially in the busi- ness world, so charting in Excel is a skill worth mastering. We illustrate some of the possibilities here, but we urge you to experiment with other possibilities on your own. Start by selecting the range E18:F25—yes, including the labels in row 18. Next, click the Line dropdown list on the Insert ribbon and select the Line with Markers type. You instantly get the line chart you want, with one series for Cherry and another for Oak. Also, when the chart is selected (that is, it has a border around it), you see two Chart Tools tabs, Design and Format. There are also three useful buttons to the right of the chart. (These three buttons were intro- duced in Excel 2013, and the two tabs condense the tools in the three tabs from Excel 2007 and 2010.) The most important button is the Select Data button on the Design ribbon. It lets you choose the ranges of the data for charting in case Excel’s default choices aren’t what you want. (The default choices are based on the selected range when you create the chart.)

Click Select Data now to obtain the dialog box in Figure 1.8. On the left, you control the series (one series or multiple series) being charted; on the right, you control the data used for the horizontal axis. By selecting E18:F25, you have the series on the left correct, including the names of these series (Cherry and Oak), but if you didn’t, you could select one of the series and click the Edit button to change it. The data on the horizontal axis are currently the default 1, 2, and so on. To use the data in column A, click the Edit button on the right and select the range A19:A25. Then you can experiment with various formatting options to make the chart even better. For example, we rescaled the vertical axis to start at $300 rather than $0 (right-click any of the numbers on the vertical axis and select Format Axis), and we added a chart title at the top and a title for the horizontal axis at the bottom.

1 4 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

The many chart options are easily accessible from the Chart Tools tabs that are visible when a chart is selected. Don’t be afraid to experiment with them to produce professional-looking charts.

Figure 1.8 Select Data Source Dialog Box

The Select Data Source dialog box in Excel for Mac has a different layout, but

the options are basically the same.

The three buttons to the right of the chart don’t appear in Excel for Mac.

09953_ch01_ptg01_001-036.indd 14 04/03/19 10:51 PM

Using the Model for What-If Questions The model in Figure 1.7 can now be used to answer many what-if questions. In fact, many models are built for the purpose of permitting experimentation with various scenarios. The important point is that the model has been built in such a way that a manager can enter any desired values in the input cells, and all outputs, including the chart, will update automatically.

1-3 Introduction to Spreadsheet Modeling 1 5

1-3c Breakeven Analysis Many business problems require you to find the appropriate level of some activity. This might be the level that maximizes profit (or minimizes cost), or it might be the level that allows a company to break even—no profit, no loss. The following example illustrates a typical breakeven analysis.

EXAMPLE

1.3 BREAKEVEN ANALYSIS AT QUALITY SWEATERS The Quality Sweaters Company sells hand-knitted sweaters. The company is planning to print a catalog of its products and undertake a direct mail campaign. The cost of printing the catalog is $20,000 plus $0.10 per catalog. The cost of mailing each catalog (including postage, order forms, and buying names from a mail-order database) is $0.15. In addition, the company plans to include direct reply envelopes in its mailings and incurs $0.20 in extra costs for each direct mail envelope used by a respondent. The average size of a customer order is $40, and the company’s variable cost per order (primarily due to labor and material costs) averages about 80% of the order’s value—that is, $32. The company plans to mail 100,000 catalogs. It wants to develop a spreadsheet model to answer the following questions:

1. How does a change in the response rate affect profit? 2. For what response rate does the company break even?

Objective To learn how to work with range names, to learn how to answer what-if questions with one-way data tables, to introduce Excel’s Goal Seek tool, and to learn how to document and audit Excel models with cell comments and Excel’s formula auditing tools.

Solution The completed spreadsheet model appears in Figure 1.9. (See the file Breakeven Analysis Finished.xlsx.) First, note the clear layout of the model. The input cells are colored blue, they are separated from the outputs, headings are boldfaced, several headings are indented, numbers are formatted appropriately, and a list to the right spells out all range names we have used. (See the next Excel Tip on how to create this list.) Also, following the convention we use throughout the book, the decision variable (number mailed) is colored red, and the bottom-line output (profit) is colored gray.

Creating Range Names

To create a range name for a range of cells (which could be a single cell), highlight the cell(s), click in the Name Box just to the left of the Formula Bar, and type a range name. Alternatively, if a column (or row) of labels appears next to the cells to be range-named, you can use these labels as the range names. To do this, highlight the labels and the cells to be named (for example, A4:B5 in Figure 1.9), click Create from Selection on the Formulas ribbon, and make sure the appropriate box in the resulting dialog box is checked. The labels in our example are to the left of the cells to be named, so the Left column box should be checked. This is a quick way to create range names, and we did it for all range names in the example. Note that if a label contains any “illegal” range-name characters, such as a space, the illegal characters are converted to underscores.

Excel Tip

Adopt some layout and formatting conventions, even if they differ from ours, to make your spreadsheets readable and easy to follow.

09953_ch01_ptg01_001-036.indd 15 04/03/19 10:51 PM

To create this model, you can proceed through the following steps.

1. Headings and range names. We have named a lot of cells, more than you might want to name, but you will see their value when you create formulas.

2. Values of input variables and the decision variable. Enter these values and format them appropriately. As usual, we have used our blue/red/gray color-coding scheme. Note that the number mailed has been designated as a decision variable, not as an input variable (and it is colored red, not blue). This is because the company gets to choose the value of this vari- able. Finally, some of the values have been combined in the statement of the problem. For example, the $32.20 in cell B12 is really 80% of the $40 average order size, plus the $0.20 per return envelope. To document this, comments appear in a few cells, as shown in Figure 1.10.

1 6 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

1 2 3 4 5 6 7 8 9

10 11 12 13

IHGFEDCBA Quality Sweaters direct mail model Range names used

11$B$!ledoM=redro_egarevA Catalog ledoMstupni of responses Fixed_cost_of_printing =Model!$B$4 Fixed cost of printing $20,000 Response 8$B$!ledoM=deliam_rebmuN%8etar Variable cost of printing mailing $0.25 Number of 5$E$!ledoM=sesnopser_fo_rebmuN0008sesnopser

31$E$!ledoM=tiforP Decision ledoMelbairav of revenue, costs, and profit 4$E$!ledoM=etar_esnopseR Number latoT000001deliam 21$E$!ledoM=tsoc_latoT000,023$euneveR

Fixed cost of printing $20,000 Total_Revenue =Model!$E$8 Order inputs Total variable cost of printing mailing $25,000 Variable_cost_of_printing_mailing =Model!$B$5 Average latoT04$redro variable cost of orders $257,600 Variable_cost_per_order =Model!$B$12 Variable cost per order $32.20 Total 006,203$tsoc

004,71$tiforP

Trial value, will do sensitivity analysis on

Includes $0.10 for printing and $0.15 for mailing each catalog

Includes 80% of the average $40 order size, plus $0.20 per return envelope

Figure 1.10 Cell Comments in Model

Figure 1.9 Quality Sweaters Model

1 2 3 4 5 6 7 8 9

10 11 12 13

IHGFEDCBA Quality Sweaters direct mail model Range names used

31$E$!ledoM=tiforP Decision ledoMelbairav of revenue, costs, and profit 4$E$!ledoM=etar_esnopseR Number latoT000001deliam 21$E$!ledoM=tsoc_latoT000,023$euneveR

004,71$tiforP

Pasting Range Names

Including a list of the range names in your spreadsheet is useful for documentation. To do this, select a cell (such as cell G2 in Figure 1.9), select the Use in Formula dropdown list from the Formulas ribbon, and then click the Paste List option. You get a list of all range names and their cell addresses. However, if you change any of these range names (delete one, for example), the paste list does not update automatically; you have to create it again.

Excel Tip

Inserting Cell Comments

Inserting comments in cells is a great way to document non-obvious aspects of your spreadsheet models. To enter a comment in a cell, right-click the cell, select the Insert Comment item, and type your comment. This creates a little red mark in the cell, indicating a comment, and you can see the comment by moving the cursor over the cell. When a cell contains a comment, you can edit or delete the comment by right-clicking the cell and selecting the appropriate item.

Excel Tip

This option to paste a list of range names is evidently not available in Excel for Mac.

09953_ch01_ptg01_001-036.indd 16 04/03/19 10:51 PM

3. Model the responses. Enter any reasonable value, such as 8%, in the Response_rate cell. You will perform sensitivity on this value later on. Then enter the formula

5Number_mailed*Response_rate

in cell E5. (Are you starting to see the advantage of range names?)

4. Model the revenue, costs, and profits. Enter the following formulas in cells E8 through E13:

5Number_of_responses*Average_order 5Fixed_cost_of_printing 5Variable_cost_of_printing_mailing*Number_mailed 5Number_of_responses*Variable_cost_per_order 5SUM(E9:E11) 5Total_revenue-Total_cost

These formulas should all be self-explanatory because of the range names used.

Entering Formulas with Range Names

To enter a formula that contains range names, you do not have to type the full range names. You actually have two options. First, you can click or drag the cells, and range names will appear in your formulas. Second, you can start typing the range name in the formula and, after a few letters, Excel will show you a list you can choose from.

Excel Tip

Forming a One-Way Data Table Now that a basic model has been created, the questions posed by the company can be answered. For question 1, you can form a one-way data table to show how profit varies with the response rate as shown in Figure 1.11. Data tables are used often in this book, so make sure you understand how to create them. First, enter a sequence of trial values of the response rate in column A, and enter a link to profit in cell B17 with the formula 5Profit. This cell is shaded for emphasis, but this isn’t necessary. (In general, other outputs could be part of the table, and they would be placed in columns C, D, and so on. There would be a link to each output in row 17.) Finally, highlight the entire table range, A17:B27, and select Data Table from the What-If Analysis dropdown list on the Data ribbon to bring up the dialog box in Figure 1.12. Fill it in as shown to indicate that the only input, Response_rate, is listed along a column. (You can enter either a range name or a cell address in this dialog box. The easiest way is to point to the cell.)

When you click OK, Excel substitutes each response rate value in the table into the Response_rate cell, recalculates profit, and reports it in the table. For a final touch, you can create a chart of the values in the data table. To do this, select the A18:B27 range and select the type of chart you want from the Insert ribbon. Then you can fix it up by adding titles and making other modifications to suit your taste.

Data tables are also called what-if tables. They let you see what happens to selected outputs as selected inputs change.

15 16 17 18 19 20 21 22 23 24 25 26 27

FEDCBA Question 1 - sensitivity of profit to response rate

Response rate Profit $17,400

1% –$37,200 2% –$29,400 3% –$21,600 4% –$13,800 5% –$6,000 6% $1,800 7% $9,600 8% $17,400 9% $25,200

10% $33,000

$40,000 $30,000 $20,000 $10,000

–$10,000 –$20,000 –$30,000 –$40,000 –$50,000

$0 0% 2% 4% 6% 8% 10% 12%

Profit versus Response Rate

Pr of

Response Rate

Figure 1.11 Data Table and Chart for Profit

1-3 Introduction to Spreadsheet Modeling 1 7

The second option below doesn’t appear to work in Excel for Mac.

09953_ch01_ptg01_001-036.indd 17 04/03/19 10:51 PM

Figure 1.12 Data Table Dialog Box

the power of Data tables

Unfortunately, many Excel users are unaware of data tables, which are among the most powerful and useful tools Excel has to offer. After you have developed a model that relates inputs to outputs, you can then build data tables in a matter of seconds to see how the outputs vary as key inputs vary over some range. Data tables are perfect for answering a large number of what-if questions quickly and easily.

Fundamental Insight

As the chart indicates, profit increases in a linear manner as the response rate varies. More specifically, each percentage point increase in the response rate increases profit by $7800. Here is the reasoning. Each percentage point increase in response rate results in 100,000(0.01) 5 1000 more orders. Each order yields a revenue of $40, on average, but incurs a variable cost of $32.20. The net gain in profit is $7.80 per order, or $7800 for 1000 orders.

Using Goal Seek From the data table, you can see that profit changes from negative to positive when the response rate is somewhere between 5% and 6%. Question 2 asks for the exact breakeven point. You could find this by trial and error, but it is easier to use Excel’s Goal Seek tool. Essentially, Goal Seek is used to solve a single equation in a single unknown. Here, the equation is Profit 5 0, and the unknown is the response rate. To implement Goal Seek, select Goal Seek from the What-If Analysis dropdown list on the Data ribbon and fill in the resulting dialog box as shown in Figure 1.13. (Range names or cell addresses can be used in the top and bottom boxes, but a number must be entered in the middle box.) After you click OK, the Response_rate and Profit cells have values 5.77% and $0. In words, if the response rate is 5.77%, Quality Sweaters breaks even. If the response rate is greater than 5.77%, the company makes money; if the rate is less than 5.77%, the company loses money. However, this assumes that the company mails 100,000 catalogs. If it sends more or fewer catalogs, the breakeven response rate will change.

The purpose of the Goal Seek tool is to solve one equation in one unknown. It is used here to find the response rate that makes profit equal to 0.

1 8 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

One-Way Data Table

A one-way data table allows you to see how one or more outputs vary as a single input varies over a selected range of values. These input values can be arranged vertically in a column or horizontally in a row. We explain only the vertical arrangement because it is the most common. To create the table, enter the input values in a column range, such as A18:A27 of Figure 1.11, and enter links to one or more output cells in columns to the right and one row above the inputs, as in cell B17. Then select the entire table, beginning with the upper-left blank cell (A17), select Data Table from the What-If Analysis dropdown list on the Data ribbon, and fill in the resulting dialog box as in Figure 1.12. Leave the Row Input cell blank and use the cell where the original value of the input lives as the Column Input cell. When you click OK, each value in the left column of the table is substituted into the column input cell, the spreadsheet recalculates, and the resulting value of the output is placed in the table. Also, if you click anywhere in the body of the table (B18:B27 in the figure), you will see that Excel has entered the TABLE function to remind you that a data table lives here. Note that the column input cell must be on the same worksheet as the table itself; otherwise, Excel issues an error.

Excel Tool

09953_ch01_ptg01_001-036.indd 18 04/03/19 10:51 PM

Figure 1.13 Goal Seek Dialog Box

Goal Seek

The purpose of the Goal Seek tool is to solve one equation in one unknown. Specifically, Goal Seek allows you to vary a single input cell to force a single output cell to a selected value. To use it, select Goal Seek from the What-If Analysis dropdown list on the Data ribbon and fill in the resulting dialog box in Figure 1.13. Enter a reference to the output cell in the Set cell box, enter the numeric value you want the output cell to equal in the To value box, and enter a reference to the input cell in the By changing cell box. Note that Goal Seek sometimes stops when the Set cell is close, but not exactly equal to, the desired value. To improve Goal Seek’s accuracy, you can select Options from the File menu and then the Formulas link. Then you can check the Enable iterative calculation box and reduce Maximum Change to any desired level of precision. We chose a precision level of 0.000001. For this level of precision, Goal Seek searches until profit is within 0.000001 of the desired value, $0.

Excel Tool

The Formula Auditing tool is indispensable for untangling the logic in a spreadsheet, especially if someone else developed it and you are trying to figure out what they did.

Using the Formula Auditing Tool The model in this example is fairly small and simple. Even so, you can use a handy Excel tool to see how all the parts are related. This is the Formula Auditing tool, which is available on the Formulas ribbon. See Figure 1.14.

The Trace Precedents and Trace Dependents buttons are probably the most useful buttons in this group. To see which formulas have direct links to the Number_of_ responses cell, select this cell and click the Trace Dependents button. Arrows are drawn to each cell that directly depends on the number of responses, as shown in Figure 1.15. Alternatively, to see which cells are used to create the formula in the Total_ revenue cell, select this cell and click the Trace Precedents button. Now you see that the Average_order and Number_of_responses cells are used directly to calculate revenue, as shown in Figure 1.16. Using these two buttons, you can trace your logic (or someone else’s logic) as far backward or forward as you like. When you are finished, just click the Remove Arrows button.

Figure 1.15 Dependents of Number_of_responses Cell 1

2 3 4 5 6 7 8 9

10 11 12 13

EDCBA Quality Sweaters direct mail model

Catalog ledoMstupni of responses Fixed cost of printing $20,000 Response %8etar Variable cost of printing mailing $0.25 Number of 0008sesnopser

Decision ledoMelbairav of revenue, costs, and profit Number latoT000001deliam 000,023$euneveR

Fixed cost of printing $20,000 Order inputs Total variable cost of printing mailing $25,000 Average latoT04$redro variable cost of orders $257,600 Variable cost per order $32.20 Total 006,203$tsoc

004,71$tiforP

Figure 1.14 Formula Auditing Group

1-3 Introduction to Spreadsheet Modeling 1 9

Change the Goal Seek accuracy in Excel for Mac by selecting Preferences from the Excel menu and then choosing the Calculation group.

09953_ch01_ptg01_001-036.indd 19 04/03/19 10:51 PM

Figure 1.16 Precedents of Total_revenue Cell 1

2 3 4 5 6 7 8 9

10 11 12 13

EDCBA Quality Sweaters direct mail model

Catalog ledoMstupni of responses Fixed cost of printing $20,000 Response %8etar Variable cost of printing mailing $0.25 Number of 0008sesnopser

Decision ledoMelbairav of revenue, costs, and profit Number latoT000001deliam 000,023$euneveR

Fixed cost of printing $20,000 Order inputs Total variable cost of printing mailing $25,000 Average latoT04$redro variable cost of orders $257,600 Variable cost per order $32.20 Total 006,203$tsoc

004,71$tiforP

1-3d Ordering with Quantity Discounts and Demand Uncertainty In the following example, we again attempt to find the appropriate level of some activity: how much of a product to order when customer demand for the product is uncertain. Two important features of this example are the presence of quantity discounts and the explicit use of probabilities to model uncertain demand.

Formula Auditing Toolbar

The formula auditing toolbar allows you to see dependents of a selected cell (which cells have formulas that reference this cell) or precedents of a given cell (which cells are referenced in this cell’s formula). You can even see dependents or precedents that reside on a different worksheet. In this case, the auditing arrows appear as dashed lines and point to a small spreadsheet icon. By double-clicking the dashed line, you can see a list of dependents or precedents on other worksheets. These tools are especially useful for understanding how someone else’s spreadsheet works.

Excel Tool

2 0 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

EXAMPLE

1.4 ORDERING WITH QUANTITY DISCOUNTS AT SAM’S BOOKSTORE Sam’s Bookstore, with many locations across the United States, places orders for all of the latest books and then distributes them to its individual bookstores. Sam’s needs a model to help it order the appropriate number of any title. For example, Sam’s plans to order a popular new hardback novel, which it will sell for $30. It can purchase any number of this book from the publisher, but due to quantity discounts, the unit cost for all books it orders depends on the number ordered. If the number ordered is less than 1000, the unit cost is $24. After each 1000, the unit cost drops: to $23 for at least 1000 copies; to $22.25 for at least 2000; to $21.75 for at least 3000; and to $21.30 (the lowest possible unit cost) for at least 4000. For example, if Sam’s orders 2500 books, its total cost is $22.25(2500) 5 $55,625. Sam’s is uncertain about the demand for this book—it estimates that demand could be anywhere from 500 to 4500. Also, as with most hardback novels, this one will eventually come out in paperback. Therefore, if Sam’s has any hardbacks left when the paperback comes out, it will put them on sale for $10, at which price it believes all leftovers will be sold. How many copies of this hardback novel should Sam’s order from the publisher?

09953_ch01_ptg01_001-036.indd 20 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 2 1

Objective To learn how to build in complex logic with IF formulas, to get help about Excel functions, to learn how to use lookup functions, to see how two-way data tables provide answers to more extensive what-if questions, and to learn about Excel’s SUMPRODUCT function.

Solution The profit model appears in Figure 1.17. (See the file Quantity Discounts Finished.xlsx.) The order quantity and demand in the Order_quantity and Demand cells are trial values. (Comments in these cells are a reminder of this.) You can enter any values in these cells to test the logic of the model. The Order_quantity cell is colored red because the company can choose its value. In contrast, the Demand cell is colored green to indicate that this input value is uncertain and is being treated explicitly as such. Also, a table is used to indicate the quantity discounts cost structure. You can use the following steps to build the model.

1 A B C D E F G H I J K

Ordering decision with quantity egnaRstnuocsid names used: 2 3 4 5 6 7 8 9

10 11 12

!$ledoM=tsoC B$18 ytitnauQstupnI discount structure 9$E$:5$D$!ledoM=pukooLtsoC

Unit cost - see table to tAthgir least Unit 21$B$!ledoM=dnameDtsoc Regular 6$B$!ledoM=ecirp_revotfeL00.42$003$ecirp Leftover 9$B$!ledoM=ytitnauq_redrO00.32$000101$ecirp

53$J$:53$B$!ledoM=seitilibaborP52.22$0002 Decision variable 91$B$!ledoM=tiforP57.12$0003 Order 5$B$!ledoM=ecirp_ralugeR03.12$00040052ytitnauq

71$B$!ledoM=euneveR Uncertain quantity Units_sold_at_leftover_price =Model!$B$16

Units_sold_at_regular_price =Model!$B$150002dnameD 13 14 15 16 17 18 19

Profit model Units sold at regular price 2000 Units sold at leftover price 500

000,56$euneveR 526,55$tsoC 573,9$tiforP

Figure 1.17 Sam’s Profit Model

1. Inputs and range names. Enter all inputs and name the ranges as indicated. The Create from Selection shortcut was used to name all ranges except for CostLookup and Probabilities. For these latter two, you can select the ranges and enter the names in the Name Box—the “manual” method.

2. Revenues. The company can sell only what it has, and it sells any leftovers at the discounted sale price. Therefore, enter the following formulas in cells B15, B16, and B17:

5MIN(Order_quantity,Demand) 5IF(Order_quantity>Demand, Order_quantity-Demand,0) 5Units_sold_at_regular_price*Regular_price1Units_sold_at_leftover_price*Leftover_price

The logic in the first two of these cells is necessary to account correctly for the cases when the order quantity is greater than demand and when it is less than or equal to demand. You could use the following equivalent alternative to the IF function in cell B16:

5 MAX(Order_quantity-Demand,0)

fx Button and Function Library Group

To learn more about an Excel function, click the fx button next to the Formula bar. This is called the Insert Function button, although some people call it the Function Wizard. If there is already a function, such as an IF function, in a cell and you then click the fx button, you will get help on this function. If you select an empty cell and then click the fx button, you can choose a function to get help on. (The same help is available from the Function Library group on the Formulas ribbon.)

Excel Tool The fx button in Excel for Mac opens a Formula Builder pane to the right, but the functionality is essentially the same as in Excel for Windows.

09953_ch01_ptg01_001-036.indd 21 04/03/19 10:51 PM

2 2 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

3. Total ordering cost. Depending on the order quantity, you can find the appropriate unit cost from the unit cost table and multiply it by the order quantity to obtain the total ordering cost. This can be accomplished with a complex nested IF for- mula, but a better way is to use the VLOOKUP function. Specifically, enter the formula

5VLOOKUP(Order_quantity,CostLookup,2)*Order_quantity

in cell B18. The VLOOKUP part of this formula says to compare the order quantity to the first (leftmost) column of the table in the CostLookup range and return the corresponding value in the second column (because the third argument is 2).

VLOOKUP

The VLOOKUP function is one of Excel’s most useful functions. To use it, first create a vertical lookup table, with values to use for comparison listed in the left column of the table and corresponding output values in as many columns to the right as you like. (See the CostLookup range in Figure 1.17 for an example.) Then the VLOOKUP function takes three or four arguments: (1) the value you want to compare to the values in the left column of the table; (2) the lookup table range; (3) the index of the column you want the returned value to come from, where the index of the left column is 1, the index of the next column is 2, and so on; and optionally (4) TRUE (for an approximate match, the default) or FALSE (for an exact match). If you omit the last argument, the values in the left column of the table must be entered in ascending order. (See online help for more details.) If the last argument is TRUE or is omitted, Excel scans down the leftmost column of the table and finds the last entry less than or equal to the first argument. (In this sense, it finds an approximate match.) There is also an HLOOKUP function that works exactly the same way, except that the lookup table is arranged in rows, not columns.

Excel Function

4. Profit. Calculate the profit with the formula

5Revenue-Cost

Two-Way Data Table The next step is to create a two-way data table for profit as a function of the order quantity and demand (see Figure 1.18). To create this table, first enter a link to the profit with the formula 5Profit in cell A22, and enter possible order quantities and possible demands in column A and row 22, respectively. (We used the same values for both order quantity and demand, from 500 to 4500 in increments of 500. This is not necessary—the demand could change in increments of 100 or even 1—but it is reasonable. Perhaps Sam’s is required by the publisher to order in multiples of 500.) Then select Data Table from the What-If Analysis dropdown list on the Data ribbon, and enter the Demand cell as the Row Input cell and the Order_quantity cell as the Column Input cell.

A two-way data table allows you to see how a single output varies as two inputs vary simultaneously.

21 A B C D E F G H I J

Data table of profit as a function of order quantity (along side) and demand (along top) 22 23 24 25 26 27 28 29 30

$9,375 500 1000 1500 2000 2500 3000 3500 4000 4500 500 $3,000 $3,000 $3,000 $3,000 $3,000 $3,000 $3,000 $3,000 $3,000

1000 -$3,000 $7,000 $7,000 $7,000 $7,000 $7,000 $7,000 $7,000 $7,000 1500 -$9,500 $500 $10,500 $10,500 $10,500 $10,500 $10,500 $10,500 $10,500 2000 -$14,500 -$4,500 $5,500 $15,500 $15,500 $15,500 $15,500 $15,500 $15,500 2500 -$20,625 -$10,625 -$625 $9,375 $19,375 $19,375 $19,375 $19,375 $19,375 3000 -$25,250 -$15,250 -$5,250 $4,750 $14,750 $24,750 $24,750 $24,750 $24,750 3500 -$31,125 -$21,125 -$11,125 -$1,125 $8,875 $18,875 $28,875 $28,875 $28,875 4000 -$35,200 -$25,200 -$15,200 -$5,200 $4,800 $14,800 $24,800 $34,800 $34,800

31 4500 -$40,850 -$30,850 -$20,850 -$10,850 -$850 $9,150 $19,150 $29,150 $39,150

Figure 1.18 Profit as a Function of Order Quantity and Demand

09953_ch01_ptg01_001-036.indd 22 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 2 3

Two-Way Data Table

A two-way data table allows you to see how a single output cell varies as you vary two input cells. Unlike a one- way data table, only a single output cell can be used. To create this type of table, enter a reference to the output cell in the top-left corner of the table, enter possible values of the two inputs below and to the right of this corner cell, and select the entire table. Then select Data Table from the What-If Analysis dropdown on the Data ribbon, and enter references to the cells where the original two input variables live. The Row Input cell corresponds to the values along the top row of the table, and the Column Input cell corresponds to the values along the left-most column of the table. When you click OK, Excel substitutes each pair of input values into these two input cells, recalculates the spreadsheet, and enters the corresponding output value in the table.

Excel Tool

SUMPRODUCT

The SUMPRODUCT function takes two range arguments, which must be exactly the same size and shape, and it sums the products of the corresponding values in these two ranges. For example, the formula 5SUMPRODUCT(A10:B11,E12:F13) is a shortcut for a formula involving the sum of four products: 5A10*E12 1A11*E13 1B10*F12 1B11*F13. This is an extremely useful function, especially when the rang- es involved are large, and it is used repeatedly throughout the book. (Actually, the SUMPRODUCT function can have more than two range arguments, all the same size and shape, but the most common use of SUMPRODUCT is when only two ranges are involved.)

Excel Function

The resulting data table shows that profit depends heavily on both order quantity and demand and (by scanning across rows) how higher demands lead to larger profits. But it is still unclear which order quantity Sam’s should select. Remember that Sam’s can choose the order quantity (the row of the data table), but it has no direct control over demand (the column of the table).

The ordering decision depends not only on which demands are possible but also on which demands are likely to occur. The usual way to express this information is with a set of probabilities that sum to 1. Suppose Sam’s estimates these as the values in row 35 of Figure 1.19. These estimates are probably based on other similar books it has sold in the past. The most likely demands are 2000 and 2500, with other values on both sides less likely. You can use these probabilities to find an expected profit for each order quantity. This expected profit is a weighted average of the profits in any row in the data table, using the probabilities as the weights. The easiest way to do this is to enter the formula

5SUMPRODUCT(B23:J23,Probabilities)

in cell B38 and copy it down to cell B46. You can also create a chart of these expected profits, as shown in Figure 1.19. (Excel refers to these as column charts. The height of each bar is the expected profit for that particular order quantity.)

This is actually a preview of decision making under uncertainty. To calculate an expected profit, you multiply each profit by its probability and add the products.

The largest of the expected profits, $12,250, corresponds to an order quantity of 2000, so we would recommend that Sam’s order 2000 copies of the book. This does not guarantee that Sam’s will make a profit of $12,250—the actual profit depends on the eventual demand—but it represents a reasonable way to proceed in the face of uncertain demand.

09953_ch01_ptg01_001-036.indd 23 04/03/19 10:51 PM

33 A B C D E F G H I J K

Model of expected demands 34 35 36 37 38 39 40 41 42 43 44

00540004005300030052000200510001005dnameD 510.040.070.051.052.052.051.050.0520.0ytilibaborP

Sum of probabilities --> 1 Order quantity Expected profit

500 $3,000 1000 $6,750 1500 $9,500 2000 $12,250 2500 $11,375 3000 $9,500 3500 $4 875

Order 2000 to maximize the expected profit.

45 46 47 48 49 50 51

4000 $1,350 4500 -$4,150 Ex

pe ct

ed P

ro fit

1 2 3 4 5 6 7 8

Expected Profit versus Order Quantity

Order Quantity

$6,000 $8,000

$10,000 $12,000 $14,000

-$6,000 -$4,000 -$2,000

$0 $2,000 $4,000

Figure 1.19 Comparison of Expected Profits

2 4 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

1-3e Estimating the Relationship Between Price and Demand The following example illustrates a very important modeling concept: estimating relation- ships between variables by curve fitting. The ideas can be illustrated at a relatively low level by taking advantage of some useful Excel tools.

EXAMPLE

1.5 ESTIMATING SENSITIVITY OF DEMAND TO PRICE AT THE LINKS COMPANY The Links Company sells its golf clubs at golf outlet stores throughout the United States. The company knows that demand for its clubs varies considerably with price. In fact, the price has varied over the past 12 months, and the demand at each price level has been observed. The data are in the data sheet of the file Golf Club Demand.xlsx (see Figure 1.20). For example, during month 12, when the price was $390, 6800 sets of clubs were sold. (The demands in column C are in hundreds of units. The cell comment in cell C3 is a reminder of this.) The company wants to estimate the relationship between demand and price and then use this estimated relationship to answer the following questions:

1. Assuming the unit cost of producing a set of clubs is $250 and the price must be a multiple of $10, what price should Links charge to maximize its profit?

2. How does the optimal price depend on the unit cost of producing a set of clubs?

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15

A B C Demand for golf clubs

Month Price Demand 1 450 45 2 300 103 3 440 49 4 360 86 5 290 125 6 450 52 7 340 87 8 370 68 9 500 45

10 490 44 11 430 58 12 390 68

Figure 1.20 Demand and Price Data for Golf Clubs

09953_ch01_ptg01_001-036.indd 24 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 2 5

Objective To illustrate Excel’s Trendline tool, and to illustrate conditional formatting.

Solution This example is divided into two parts: estimating the relationship between price and demand, and creating the profit model. (Both can be found in the file Golf Club Demand Finished.xlsx.)

Estimating the Relationship Between Price and Demand A scatterplot of demand versus price appears in Figure 1.21. (This can be created in the usual way as an Excel chart of the scatter type.) Obviously, demand decreases as price increases, but the goal is to quantify this relationship. Therefore, after cre- ating this chart, right-click any point on the chart and select Add Trendline to see the pane in Figure 1.22. This allows you to superimpose several different curves (including a straight line) on the chart. We consider three possibilities, the linear, power, and exponential curves, defined by the following general equations (where y and x, a general output and a general input, cor- respond to demand and price for this example):

• Linear: y 5 a 1 bx • Power: y 5 axb

• Exponential: y 5 aebx

D em

an d

Price

100

110

120

130

280 330 380 430 480 530

Figure 1.21 Scatterplot of Demand Versus Price

Figure 1.22 Trendline Options Dialog Box

09953_ch01_ptg01_001-036.indd 25 04/03/19 10:51 PM

2 6 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

Before proceeding, we describe some general properties of these three functions because of their widespread applicability. The linear function is the easiest. Its graph is a straight line. When x changes by 1 unit, y changes by b units. The constant a is called the intercept, and b is called the slope.

The power function is a curve except in the special case where the exponent b is 1. (Then it is a straight line.) Assuming that a is positive, the shape of this curve depends on the exponent b. If b 7 1, y increases at an increasing rate as x increases. If 0 6 b 6 1, y increases, but at a decreasing rate, as x increases. Finally, if b 6 0, y decreases as x increases. An important property of the power curve is that when x changes by 1%, y changes by a constant percentage, and this percentage is approx- imately equal to b%. For example, if y 5 100x 2 2.35, then every 1% increase in x leads to an approximate 2.35% decrease in y.

The exponential function also represents a curve whose shape depends on the constant b in the exponent. Again, assume that a is positive. Then if b 7 0, y increases as x increases; if b 6 0, y decreases as x increases. An important property of the exponential function is that if x changes by 1 unit, y changes by a constant percentage, and this percentage is approximately equal to 100 3 b%. For example, if y 5 100e 2 0.014x, then whenever x increases by 1 unit, y decreases by approximately 1.4%. Here, e is the special number 2.7182 . . . , and e to any power can be calculated in Excel with the EXP function. For example, you can calculate e 2 0.014 with the formula =EXP(−0.014).

Returning to the example, if you superimpose any of these curves on the chart of demand versus price, Excel chooses the best-fitting curve of that type. In addition, if you check the Display Equation on Chart option, you see the equation of this best-fitting curve. For example, the best-fitting power curve appears in Figure 1.23. (The equation might not appear exactly as in the figure. However, it can be resized and reformatted to appear as shown.)

Each of these curves provides the best-fitting member of its “family” to the demand/price data, but which of these three is best overall? You can answer this question by finding the mean absolute percentage error (MAPE) for each of the three curves. To do so, for any price in the data set and any of the three curves, first predict demand by substituting the given price into the equation for the curve. The predicted demand is typically not the same as the observed demand, so you can calculate the absolute percentage error (APE) with the general formula:

APE 5 |Observed demand 2 Predicted demand| }}}}}

Observed demand (1.1)

Then for any curve, MAPE is the average of these APE values. The curve with the smallest MAPE is the best fit overall. MAPE is a popular error measure, but it is not the only error measure used. Another popular error measure is RMSE

(root mean square error), the square root of the average of the squared errors. Another possible measure is MAE (mean absolute error), the average of the absolute errors. (MAE is sometimes called MAD, mean absolute deviation.) These three measures often, but not always, give the same rankings of fits, so any of them can be used, and all three are calculated in this example. However, the advantage of MAPE is that it is easy to interpret a value such as 5.88%: the fitted values are off, on average, by 5.88%. The values of RMSE and MAE are more difficult to interpret because they depend on the units of the observations.

The calculations appear in Figure 1.24. (You can check the formulas in the finished version of the file.) As you can see, the three error measures provide the same rankings, with the power fit being best according to all three. In particular, MAPE for the power curve is 5.88%, meaning that its predictions are off, on average, by 5.88%. This power curve predicts that each 1% increase in price leads to an approximate 1.9% decrease in demand.

110

100

120

130

40 280 330 380

Price

D em

an d

Power Fit

430 480 530

y = 6E+06x -1.908

Figure 1.23 Best-Fitting Power Curve

09953_ch01_ptg01_001-036.indd 26 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 2 7

Developing the Profit Model Now we move to the profit model, which assumes the company produces exactly enough sets of clubs to meet demand. This model, shown in Figure 1.25, is straightforward to develop; you can see the details in the finished version of the file.

Maximizing Profit To see which price maximizes profit, you can build the data table shown in Figure 1.26. Here, the column input cell is B11 and the linking formula in cell B25 is 5B17. The corresponding chart shows that profit first increases and then decreases. You can find the maximum profit and corresponding price in at least three ways. First, you can estimate them from the chart. Second, you can scan down the data table for the maximum profit, which is shown in the figure. The following Excel Tip describes a third method that uses some of Excel’s more powerful features.

17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

A B C D E F G H I J Parameters of best-fitting curves

laitnenopxErewoPraeniLlaitnenopxErewoPraeniLraeniL %87.31%98.21%89.4102.1508.0547.1513.112tpecretnI

Slope - %38.3%19.6%78.149.60121.01139.4016453.0 %57.9%12.8%38.2187.3520.3592.55rewoP %83.7%85.9%37.256.9767.7756.384601785tnatsnoC

Exponent - %41.01%10.6%22.3123.21184.71184.8012809.1 %35.1%13.2%05.002.1508.0547.15laitnenopxE %00.1%23.0%13.478.7837.6857.0915.664tnatsnoC

Exponent - %25.11%35.8%18.7148.5708.3711.0819400.0 34.01 41.55 40.06 24.42% 7.67% 10.99% 37.56 43.18 42.07 14.65% 1.86% 4.38% 58.83 55.40 56.49 1.43% 4.48% 2.61% 73.02 66.75 68.74 7.38% 1.84% 1.09%

etulosbAnoitciderP percentage error K

Linear 45.43

3.72 39.51

5.50 273.04

0.07 14.03

146.60 120.78

41.53 0.69

25.16

33.63 50.72 16.20 67.83 56.53

1.44 0.08

33.68 11.92

0.67 6.75 1.56

38.47 15.53 22.84 40.28

160.70 0.64 0.76

61.41 24.44

3.72 2.29 0.55

6.74 1.93 6.29 2.35

16.52 0.26 3.75

12.11 10.99

6.44 0.83 5.02

5.80 7.12 4.02 8.24 7.52 1.20 0.27 5.80 3.45 0.82 2.60 1.25

6.20 3.94 4.78 6.35

12.68 0.80 0.87 7.84 4.94 1.93 1.51 0.74

Power Exponential ExponentialLinear Power

L M N O P Squared error Absolute error

MAPE 9.68% 5.88% 6.50% 7.72 4.84 5.57 6.10 4.01 4.38

RMSE MAE

Figure 1.24 Finding the Best-Fitting Curve Overall

A B C D E 1 2 3 4 5 6 7

Profit model, using best fitting power curve for estimating demand

Parameters of best-fitting power curve (from Estimation sheet) Constant 5871064 Exponent –1.9082

Monetary inputs 8 9

10 11 12 13

Unit cost to produce $250

Decision variable Unit price (trial value) $400

Profit model 14 15 16 17

Predicted demand 63.601 Total revenue $25,441 Total cost $15,900

045,9$tiforP

Figure 1.25 Profit Model

Conditional Formatting

Cell B39 in Figure 1.26 is colored because it corresponds to the maximum profit in the column, but Excel’s Conditional Formatting tool can do this for you—automatically. To color the maximum profit, select the range of profits, B26:B47, click the Conditional Formatting dropdown arrow on the Home ribbon, then Top/Bottom Rules, and then Top 10 Items to bring up the dialog box in Figure 1.27. By asking for the top 1 item, the max- imum value in the range is colored. You can experiment with the many other Conditional Formatting options. This is a great tool.

Excel Tip

09953_ch01_ptg01_001-036.indd 27 04/03/19 10:51 PM

2 8 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

Figure 1.26 Profit as a Function of Price 19

A B C D E F G H I Maximum profit from data table below, with corresponding best unit price

20 21 22 23 24 25 26 27

Maximum profit $10,409 Best price 025$

Data table for Profit as a func�on of Unit price Unit price Profit

$9,540 $260 $1,447 $280 $3,769

$5 50628 29 30 31 32 33 34 35 36 37

$300 , $320 $340 $360 $8,554 $380 $9,118 $400 $9,540

$480 38 39 40 41 42 43 44 45

$420 $9,851 $440 $10,075 $460 $10,230

$10,330 $500 $10,387 $520 $10,409 $540 $10,403 $560 $10,375 $580 $10,329

46 47

$600 $10,269 $620 $10,197 $640 $10,116 $660 $10,029 $680 $9,936

$6,815 $7,805

Maximum profit. Condi�onal forma�ng is used to color the largest profit. This used to be fairly difficult, but Excel versions 2007 and later make it easy with the “Top 10” op�ons – in this case, the top 1.

$260 $310 $360 $410 $460 $510 $560 $610 $660 $710

Pr of

Price

Profit versus Price

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

What about the corresponding best price, shown in cell B21 of Figure 1.26? You could enter this manually, but wouldn’t it be nice if you could get Excel to find the maximum profit in the data table, determine the price in the cell to its left, and report it in cell B21, all automatically? This is indeed possible. Enter the formula

5INDEX(A26:A47,MATCH(B20,B26:B47,0),1)

in cell B21, and the best price appears. This formula uses two Excel functions, MATCH and INDEX. MATCH compares the first argument (the maximum profit in cell B20) to the range specified in the second argument (the range of profits) and returns the index of the cell where a match appears. (The third argument, 0, specifies that you want an exact match.) In this case, the MATCH function returns 14 because the maximum profit is in the 14th cell of the profits range. Then the INDEX function is called effectively as =INDEX(A26:A47,14,1). The first argument is the range of prices, the second is a row index, and the third is a column index. Very simply, this function says to return the value in the 14th row and first column of the prices range.

Sensitivity to Variable Cost We now return to question 2 in the example: How does the best price change as the unit variable cost changes? You can answer this question with a two-way data table. Remember that this is a data table with two inputs—one along the left side and the other across the top row—and a single output. The two inputs for this problem are unit variable cost and unit price, and the single output is profit. The corresponding data table is in the range A55:F83, shown in Figure 1.28. To develop this table, enter desired inputs in column A and row 55, enter the linking formula 5B17 in cell A55 (it always goes in the top-left corner of a two-way data table), select the entire table, select Data Table from the What-If Analysis dropdown list, and enter B8 as the Row Input cell and B11 as the Column Input cell.

As before, you can scan the columns of the data table for the maximum profits and enter them (manually) in rows 51 and 52. (Alternatively, you can use the Excel features described in the previous Excel Tip to accomplish these tasks. Take a look at the finished version of the file for details. This file also explains how conditional formatting is used to color the max- imum profit in each column of the table.) Then you can create a chart of maximum profit (or best price) versus unit cost. The chart in Figure 1.28 shows that the maximum profit decreases at a decreasing rate as the unit cost increases.

Figure 1.27 Conditional Formatting Dialog Box

09953_ch01_ptg01_001-036.indd 28 04/03/19 10:51 PM

Figure 1.28 Profit as a Function of Unit Cost and Unit Price

Other Modeling Issues The layout of the Golf Club Demand.xlsx file is fairly straightforward. However, note that instead of a single worksheet, there are two worksheets, partly for logical purposes and partly to reduce clutter. There is one worksheet for estimation of the demand function and the various scatterplots, and there is another for the profit model.

One last issue is the placement of the data tables for the sensitivity analysis. You might be inclined to put these on a sepa- rate Sensitivity worksheet. However, Excel does not allow you to build a data table on one worksheet that uses a row or column input cell from another worksheet. Therefore, you are forced to put the data tables on the same worksheet as the profit model.

1-3f Decisions Involving the Time Value of Money In many business situations, cash flows are received at different points in time, and a com- pany must determine a course of action that maximizes the “value” of cash flows. Here are some examples:

• Should a company buy a more expensive machine that lasts for 10 years or a less expensive machine that lasts for 5 years?

• What level of plant capacity is best for the next 20 years? • A company must market one of several midsize cars. Which car should it market?

To make decisions when cash flows are received at different points in time, the key concept is that the later a dollar is received, the less valuable the dollar is. For example, suppose you can invest money at a 5% annual interest rate. Then $1.00 received now is essentially equivalent to $1.05 a year from now. The reason is that if you have $1.00 now, you can invest it and gain $0.05 in interest in one year. If r 5 0.05 is the interest rate (expressed as a decimal), we can write this as

$1.00 now 5 $1.05 a year from now 5 $1.00(1 1 r) (1.2)

Dividing both sides of Equation (1.2) by 1 1 r, this can be rewritten as

$1.00 3 1>(1 1 r) now 5 $1.00 a year from now (1.3) The value 1>(1 1 r) in Equation (1.3) is called the discount factor, and it is always less than 1. The quantity on the left, which evaluates to $0.952 for r 5 0.05, is called the pres- ent value of $1.00 received a year from now. The idea is that if you had $0.952 now, you could invest it at 5% and have it grow to $1.00 in a year.

1-3 Introduction to Spreadsheet Modeling 2 9

09953_ch01_ptg01_001-036.indd 29 04/03/19 10:51 PM

3 0 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

In general, if money can be invested at annual rate r compounded each year, then $1 received t years from now has the same value as 1>(1 1 r)t dollars received today— that is, the $1 is discounted by the discount factor raised to the t power. If you multiply a cash flow received t years from now by 1>(1 1 r)t to obtain its present value, the total of these present values over all years is called the net present value (NPV) of the cash flows. Financial theory states that projects with positive NPVs increase the value of the company, whereas projects with negative NPVs decrease the value of the company.

The rate r (usually called the discount rate) used by major corporations is gener- ally based on their weighted average cost of capital. The value of r used to evaluate any particular project depends on a number of things and can vary from project to project. Because this is the focus of finance courses, we will not pursue it here. But given a suit- able value of r, the following example illustrates how spreadsheet models and the time value of money can be used to make complex business decisions.

the time Value of Money

Money earned in the future is less valuable than money earned today, for the sim- ple reason that money earned today can be invested to earn interest. Similarly, costs incurred in the future are less “costly” than costs incurred today. This is why you typically don’t simply sum up revenues and costs in a multiperiod model. You instead discount future revenues and costs for a fair comparison with revenues and costs incurred today. The resulting sum of discounted cash flows is the net present value (NPV), and it forms the cornerstone of much of financial theory and applications.

Fundamental Insight

EXAMPLE

1.6 CALCULATING NPV AT ACRON Acron is a large drug company. At the current time, the beginning of year 0, Acron is trying to decide whether one of its new drugs, Niagra, is worth pursuing. Niagra is in the final stages of development and will be ready to enter the market one year from now. The final cost of development, to be incurred at the beginning of year 1, is $15 million. Acron estimates that the demand for Niagra will gradually grow and then decline over its useful lifetime of 20 years. Specifically, the company expects its gross margin (revenue minus cost) to be $1.5 million in year 1, then to increase at an annual rate of 6% through year 8, and finally to decrease at an annual rate of 5% through year 20. Acron wants to develop a spreadsheet model of its 20-year cash flows, assuming its cash flows, other than the initial development cost, are incurred at the ends of the respective years. (To simplify the model, taxes are ignored.) Using an annual discount rate of 7.5% for the purpose of calculating NPV, the drug company wants to answer the following questions:

1. Is the drug worth pursuing, or should Acron abandon it now and not incur the $15 million development cost? 2. How do changes in the model inputs change the answer to question 1?

Objective To illustrate efficient selection and copying of large ranges and to learn Excel’s NPV function.

Solution The model of Acron’s cash flows appears in Figure 1.29. As with many financial spreadsheet models that extend over a multi- year period, you enter “typical” formulas in the first year or two and then copy this logic down to all years. (We could make the years go across, not down. In that case, splitting the screen is useful so that you can see the first and last years of data. Splitting the screen is explained in the following Excel Tip.)

The discount factor is 1 divided by (1 plus the discount rate). To discount a cash flow that occurs t years from now, multiply it by the discount factor raised to the t power. The NPV is the sum of all discounted cash flows.

09953_ch01_ptg01_001-036.indd 30 04/03/19 10:51 PM

1-3 Introduction to Spreadsheet Modeling 3 1

Splitting the Screen

To split the screen horizontally and vertically, select the cell where you want the split to occur and click Split on the View ribbon. This creates horizontal and vertical split lines that you can move if you like.

Excel Tip

To create the model, complete the following steps. (See the file Calculating NPV Finished.xlsx.)

1. Inputs and range names. Enter the given input data in the blue cells, and name the ranges as shown. As usual, the range names for cells B4 through B9 can be created all at once with the Create from Selection shortcut, as can the range name for the gross margins in column B. In the latter case, select the range B12:B32 and then use the Create from Selection shortcut.

2. Cash flows. Start by entering the formula

5Gross_margin_year_1

in cell B13 for the year 1 gross margin. Then enter the general formula

5IF(A14<5Peak_year,B13*(11Rate_of_increase),B13*(1−Rate_of_decrease))

in cell B14 and copy it down to cell B32 to calculate the other yearly gross margins. Note how this IF function checks the year index in column A to see whether sales are still increasing or have started to decrease. By using the (range-named) input cells in this formula, you can change any of these inputs in cells B6 through B8, and the calculated cells will auto- matically update. This is a much better practice than embedding the numbers in the formula itself.

Figure 1.29 Acron’s Model of 20-Year NPV

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

A B C D E F G Calculating NPV at egnaRnorcA names used:

Development_cost =Model!$B$4 Inputs 9$B$!ledoM=etar_tnuocsiD Development Gross margin year 1 =Model!$B$5Gross_margin_year_11.2 Peak year Rate of increase Rate of decrease

cost 15 Gross_margin =Model!$B$13:$B$32

8 Peak_year =Model!$B$6 6% Rate_of_decrease =Model!$B$8 5% Rate_of_increase =Model!$B$7

Discount rate 7.5%

Cash flows End of year Gross margin

VPN

(through peak year) (after peak year)

1 1.2000 2 1.3200 3 1.4520 4 1.5972 5 1.7569 6 1.9326 7 2.1259 8 2.3385 9 2.2215

10 2.1105 11 2.0049 12 1.9047 13 1.8095 14 1.7190 15 1.6330 16 1.5514 17 1.4738 18 1.4001 19 1.3301 20 1.2636

2.3961

09953_ch01_ptg01_001-036.indd 31 04/03/19 10:51 PM

3 2 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

3. Net present value. The NPV is based on the sequence of cash flows in column B. From the general discussion of NPV, to discount everything back to the beginning of year 1, the value in cell B13 should be multiplied by 1>(1 1 r)1, the value in cell B14 should be multiplied by 1>(1 1 r)2, and so on, and these quantities should be summed to obtain the NPV. (Here, r 5 0.075 is the discount rate.) Fortunately, how- ever, Excel has a built-in NPV function to accomplish this calculation. To use it, enter the formula

5−Development_cost1NPV(Discount_rate,Gross_margin)

in cell B33. The NPV function takes two arguments: the discount rate and a range of cash flows. It assumes that the first cell in this range is the cash flow at the end of year 1, the second cell is the cash flow at the end of year 2, and so on. This explains why the development cost is subtracted outside the NPV function—it is incurred at the beginning of year 1. In general, any cash flow incurred at the beginning of year 1 must be placed outside the NPV function so that it won’t be discounted.

Note that the sum of the cash flows in column B is slightly more than $34.54 million, but the NPV (aside from the devel- opment cost) is only about $18.06 million. This is because values farther into the future are discounted so heavily. At the extreme, the $1.2188 million cash flow in year 20 is equivalent to only $1.218831>(1 1 0.075)204 5 $0.287 million now.

The stream of cash flows in the NPV function must occur at the ends of year 1, year 2, and so on. If the timing is irregular, you can discount “manually” or you can use Excel’s XNPV function.

Use the Ctrl1Enter shortcut to enter a formula in a range all at once. It is equivalent to copying.

Efficient Selection

An easy way to select a large range, assuming that the first and last cells of the range are visible, is to select the first cell and then, with your finger on the Shift key, select the last cell. (Don’t forget that you can split the screen to make these first and last cells visible when the range is large.) This selects the entire range and is easier than scrolling.

Excel Tip

Efficient Copying with Ctrl1Enter

An easy way to enter the same formula in a range all at once is to select the range (as in the preceding Excel Tip), type the formula, and press Ctrl1Enter (both keys at once). After you get used to this shortcut, you will probably use it all the time.

Excel Tip The shortcut for copying in Excel for Mac is command+enter.

NPV

The NPV function takes two arguments, the discount rate (entered as a decimal, such as 0.075 for 7.5%) and a stream of cash flows. These cash flows are assumed to occur in consecutive years, starting at the end of year 1. If there is an initial cash flow at the beginning of year 1, such as an initial investment, it should be entered outside the NPV function. (There is also an XNPV function that has three arguments: a discount rate, a series of cash flows, and a series of dates when the cash flows occur. Because these dates do not have to be equally spaced through time, this function is more flexible than the NPV function. We will not use the XNPV function in this book, but you can learn more about it in Excel’s online help.)

Excel Function

09953_ch01_ptg01_001-036.indd 32 04/03/19 10:51 PM

1-4 Conclusion In this chapter we have tried to convince you that the skills in this book are important for you as you enter the business world. The methods we discuss are no longer the sole province of the “quant jocks.” By having a computer loaded with powerful soft- ware, you incur a responsibility to use this software to analyze business problems effectively. We have also given you a taste of the spreadsheet modeling process you will see in much of the rest of the book, and you have learned a number of useful Excel tools for analyzing spreadsheet models. You will get to use these and other Excel tools in the next few chapters as you enter the exciting world of data analysis.

Deciding Whether to Continue with the Drug NPV calculations are typically used to see whether a certain project should be undertaken. If the NPV is positive, the project is worth pursuing. If the NPV is negative, the company should look for other places to invest its money. Figure 1.29 shows that the NPV for this drug is positive, over $3.06 million. Therefore, if Acron is comfortable with its predictions of future cash flows, it should continue with the development and marketing of the drug. However, Acron might first want to see how sensitive the NPV is to changes in the sales predictions. After all, these predictions are intel- ligent guesses at best.

One possible sensitivity analysis appears in Figure 1.30. This shows a one-way data table to see how the NPV changes when the number of years of increase (the input in cell B6) changes. Again, the important question is whether the NPV stays positive. It certainly does when the input variable is greater than its current value of 8. However, if sales start decreasing soon enough—that is, if the value in B6 is 4 or less—the NPV turns negative. This should probably not concern Acron, because its best guess for the years of increase is considerably greater than 4.

Another possibility is to see how long and how good the good years are. To do this, you can create the two-way data table shown in Figure 1.31, where cell B7 is the row input cell and cell B6 is the column input cell. Now you can see that if sales increase through year 6, all reasonable yearly increases result in a positive NPV. However, if sales increase only through year 5, then a low enough yearly increase can produce a negative NPV. Acron might want to step back and estimate how likely these bad scenarios are before proceeding with the drug.

11 12 13 14 15 16 17 18 19 20

D E Sensitivity to peak year (cell B6)

6.7451 3 −0.9700 4 0.5688 5 2.1146 6 3.6629 7 5.2084 8 6.7451 9 8.2657

10 9.7616

Figure 1.30 Sensitivity of NPV to Years of Sales Increase

22 23 24 25 26 27 28 29 30 31

D E F G H I J Sensitivity to rate of increase in early years (cell B7) and peak year (cell B6)

6.7451 5% 6% 7% 8% 3 4 5 6 7 8 9

–2.0305 –1.0958 –0.1995

0.6574 1.4739 2.2489 2.9809 3.6682

–1.8221 –0.7742

0.2401 1.2191 2.1609 3.0633 3.9237 4.7393

–1.6119 –0.4470

0.6911 1.8000 2.8768 3.9183 4.9206 5.8798

–1.3998 –0.1142

1.1537 2.4006 3.6227 4.8156 5.9746 7.0940

–1.1858 0.2244 1.6281 3.0214 4.3995 5.7573 7.0886 8.3864

–0.9700 0.5688 2.1146 3.6629 5.2084 6.7451 8.2657 9.7616

9% 10%

Figure 1.31 Sensitivity of NPV to Years of Increase and Yearly Increase

1-4 Conclusion 3 3

09953_ch01_ptg01_001-036.indd 33 04/03/19 10:51 PM

3 4 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

Summary of Key Terms TERM EXPLANATION EXCEL PAGES Model inputs The given numeric values in any problem

statement 21

Decision variables The variables a decision maker has control over to obtain the best solutions

Model outputs The numeric values that result from com- binations of inputs and decision variables through the use of logical formulas

IF function Useful for implementing logic =IF(condition,result_If_True, result_If_False)

relative, absolute cell addresses

Useful for copying formulas; absolute row or column stays fixed, relative row or column “moves”

A1 (relative), $A1 or A$1 (mixed), $A$1 (absolute); press F4 to cycle through possibilities

range names Useful for making formulas more readable

Type name in Name box, or use Create from Selection shortcut on Formulas ribbon

pasting range names Provides a list of all range names in the current workbook

Use Paste List from Use in Formula dropdown list on Formulas ribbon

Cell comments Useful for documenting contents of cells Right-click cell, select Insert Comment menu item

One-way data table Shows how one or more outputs vary as a single input varies

Use Data Table from What-If Analysis dropdown list on Data ribbon

Goal Seek Solves one equation in one unknown Use Goal Seek from What-If Analysis dropdown list on Data ribbon

Formula auditing toolbar Useful for checking which cells are related to other cells through formulas

Use Formula Auditing buttons on Formulas ribbon

fx button Useful for getting help on Excel functions On Formulas ribbon 41

VLOOKUp function Useful for finding a particular value based on a comparison

=VLOOKUP(valueToCompare, lookupTable,columnToReturn,True/False)

two-way data table Shows how a single output varies as two inputs vary

Use Data Table from What-If Analysis dropdown list on Data ribbon

SUMprODUCt function Calculates the sum of products of values in two (or more) same-sized ranges

=SUMPRODUCT(range1,range2) 43

trendline tool Superimposes the best-fitting line or curve of a particular type on a scatter chart or time series graph

With chart selected, right-click any point and select Add Trendline

Conditional formatting Formats cells depending on whether specified conditions hold

Use Conditional Formatting on Home ribbon

Net present value (NpV) The current worth of a stream of cash flows that occur in the future

Discount rate Interest rate used for discounting future cash flows to calculate present values

Splitting screen Useful for separating the screen horizontally and/or vertically

Select Split from View ribbon 56

efficient selection Useful for selecting a large rectangular range

While pressing the Shift key, select the upper-left and bottom-right cells of the range

efficient copying Shortcut for copying a formula to a range Select the range, enter the formula, and press Ctrl 1Enter

NpV function Calculates NPV of a stream of cash flows at the ends of consecutive years, starting in year 1

=NPV(discountRate,cashFlows) 57

09953_ch01_ptg01_001-036.indd 34 04/03/19 10:51 PM

1-4 Conclusion 3 5

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. The sensitivity analysis in Example 1.3 was on the

response rate. Suppose now that the response rate is known to be 8%, and the company wants to perform a sensitivity analysis on the number mailed. After all, this is a variable under direct control of the company. Cre- ate a one-way data table and a corresponding line chart of profit versus the number mailed, where the number mailed varies from 80,000 to 150,000 in increments of 10,000. Does it appear, from the results you see here, that there is an optimal number to mail, from all possible values, that maximizes profit?

2. Continuing the previous problem, use Goal Seek for each value of number mailed (once for 80,000, once for 90,000, and so on). For each, find the response rate that allows the company to break even. Then chart these val- ues, where the number mailed is on the horizontal axis, and the breakeven response rate is on the vertical axis. Explain the behavior in this chart.

3. In Example 1.3, the range E9:E11 does not have a range name. Open your completed Excel file and name this range Costs. Then look at the formula in cell E12. It does not automatically use the new range name. Modify the formula so that it does. Then select cell G4 and paste the new list of range names over the previous list.

4. In some ordering problems, like the one in Example 1.4, whenever demand exceeds existing inventory, the excess demand is not lost but is filled by expedited orders—at a premium cost to the company. Change Sam’s model to reflect this behavior. Assume that the unit cost of expediting is $40, well above the highest regular unit cost.

5. The spreadsheet model for Example 1.4 contains a two-way data table for profit versus order quantity and demand. Experiment with Excel’s chart types to create a chart that shows this information graphically in an intu- itive format. (Choose the format you would choose to give a presentation to your boss.)

6. In Example 1.4, the quantity discount structure is such that all the units ordered have the same unit cost. For example, if the order quantity is 2500, then each unit costs $22.25. Sometimes the quantity discount structure is such that the unit cost for the first so many items is one value, the unit cost for the next so many units is a slightly lower value, and so on. Modify the model so that Sam’s pays $24 for units 1 to 1500, $23 for units 1501 to 2500, and $22 for units 2501 and above. For example, the total cost for an order quantity of 2750 is 1500(24) 1 1000(23) 1 250(22). (Hint: Use IF func- tions, not VLOOKUP.)

7. Suppose you have an extra six months of data on demands and prices, in addition to the data in Exam- ple 1.5. These extra data points are (350,84), (385,72), (410,67), (400,62), (330,92), and (480,53). (The price is shown first and then the demand at that price.) After adding these points to the original data, use Excel’s Trendline tool to find the best-fitting linear, power, and exponential trend lines. Then calculate the MAPE for each of these, based on all 18 months of data. Does the power curve still have the smallest MAPE?

8. Consider the power curve y 5 10000x222.35. Calculate y when x 5 5; when x 5 10; and when x 5 20. For each of these values of x, find the percentage change in y when x increases by 1%. That is, find the percentage change in y when x increases from 5 to 5.05; when it increases from 10 to 10.1; and when it increases from 20 to 20.2. Is this percentage change constant? What num- ber is it very close to? Write a brief memo on what you have learned about power curves from these calcula- tions.

9. Consider the exponential curve y 5 1000e20.014x. Calcu- late y when x 5 5; when x 5 10; and when x 5 20. For each of these values of x, find the percentage change in y when x increases by one unit. That is, find the per- centage change in y when x increases from 5 to 6; when it increases from 10 to 11; and when it increases from 20 to 21. Is this percentage change constant? When expressed as a decimal, what number is it very close to? Write a brief memo on what you have learned about exponential curves from these calculations.

10. Modify the model in Example 1.6 so that development lasts for an extra year. Specifically, assume that develop- ment costs of $12 million and $3 million are incurred at the beginnings of years 1 and 2, and then the sales in the current model occur one year later, that is, from year 2 until year 21. Again, calculate the NPV discounted back to the beginning of year 1, and perform the same sensi- tivity analyses. Comment on the effects of this change in timing.

11. Modify the model in Example 1.6 so that sales increase, then stay steady, and finally decrease. Specifically, assume that the gross margin is $1.5 million in year 1, then increases by 10% annually through year 6, then stays constant through year 10, and finally decreases by 5% annually through year 20. Perform a sensitivity anal- ysis with a two-way data table to see how NPV varies with the length of the increase period (currently 6 years) and the length of the constant period (currently 4 years). Comment on whether the company should pursue the drug, given your results.

12. Create a one-way data table in the model in Example 1.6 to see how the NPV varies with discount rate, which is allowed to vary from 6% to 10% in increments of 0.5%. Explain intuitively why the results go in the direction they go—that is, the NPV decreases as the discount

09953_ch01_ptg01_001-036.indd 35 04/03/19 10:51 PM

rate increases. Should the company pursue the drug for all of these discount rates?

13. Julie James is opening a lemonade stand. She believes the fixed cost per week of running the stand is $50.00. Her best guess is that she can sell 300 cups per week at $0.50 per cup. The variable cost of producing a cup of lemonade is $0.20. a. Given her other assumptions, what level of sales vol-

ume will enable Julie to break even? b. Given her other assumptions, discuss how a change in

sales volume affects profit. c. Given her other assumptions, discuss how a change in

sales volume and variable cost jointly affect profit. d. Use Excel’s Formula Auditing tools to show which

cells in your spreadsheet affect profit directly. 14. You are thinking of opening a small copy shop. It costs

$5000 to rent a copier for a year, and it costs $0.03 per copy to operate the copier. Other fixed costs of running the store will amount to $400 per month. You plan to charge an average of $0.10 per copy, and the store will be open 365 days per year. Each copier can make up to 100,000 copies per year. a. For one to five copiers rented and daily demands of

500, 1000, 1500, and 2000 copies per day, find annual profit. That is, find annual profit for each of these combinations of copiers rented and daily demand.

b. If you rent three copiers, what daily demand for cop- ies will allow you to break even?

c. Graph profit as a function of the number of copiers for a daily demand of 500 copies; for a daily demand of 2000 copies. Interpret your graphs.

15. Dataware is trying to determine whether to give a $10 rebate, cut the price $6, or have no price change on a software product. Currently, 40,000 units of the prod- uct are sold each week for $45 apiece. The variable cost of the product is $5. The most likely case appears to be that a $10 rebate will increase sales 30%, and half of all people will claim the rebate. For the price cut, the most likely case is that sales will increase 20%. a. Given all other assumptions, what increase in sales

from the rebate would make the rebate and price cut equally desirable?

b. Dataware does not really know the increase in sales that will result from a rebate or price cut. However, the company is sure that the rebate will increase sales

by between 15% and 40% and that the price cut will increase sales by between 10% and 30%. Perform a sensitivity analysis that could be used to help deter- mine Dataware’s best decision.

16. A company manufacturers a product in the United States and sells it in England. The unit cost of manufacturing is $50. The current exchange rate (dollars per pound) is 1.221. The demand function, which indicates how many units the company can sell in England as a function of price (in pounds) is of the power type, with constant 27556759 and exponent 22.4. a. Develop a model for the company’s profit (in dollars)

as a function of the price it charges (in pounds). Then use a data table to find the profit-maximizing price to the nearest pound.

b. If the exchange rate varies from its current value, does the profit-maximizing price increase or decrease? Does the maximum profit increase or decrease?

17. The yield of a chemical reaction is defined as the ratio (expressed as a percentage) of usable output to the amount of raw material input. Suppose the yield of a chemical reaction depends on the length of time the pro- cess is run and the temperature at which the process is run. The yield can be expressed as follows:

Yield 5 90.79 2 1.095x1 2 1.045x2 2 2.781x 2 1

22.524x22 2 0.775x1x2

Here x1 5 (Temperature 2 125)/10 and x2 5 (Time 2 300)/30, where temperature is measured in degrees Fahrenheit, and time is measured in seconds. Use a data table to find the temperature and time settings that max- imize the yield of this process.

18. The payback of a project is the number of years it takes before the project’s total cash flow is positive. Pay- back ignores the time value of money. It is interesting, however, to see how differing assumptions on project growth impact payback. Suppose, for example, that a project requires a $300 million investment right now. The project yields cash flows for 10 years, and the year 1 cash flow will be between $30 million and $100 mil- lion. The annual cash flow growth will be between 5% and 25% per year. (Assume that this growth is the same each year.) Use a data table to see how the project pay- back depends on the year 1 cash flow and the cash flow growth rate.

3 6 C h a p t e r 1 I n t r o d u c t i o n t o B u s i n e s s a n a l y t i c s

09953_ch01_ptg01_001-036.indd 36 04/03/19 10:51 PM

CHAPTER 2 Describing the Distribution of a Variable

CHAPTER 3 Finding Relationships among Variables

CHAPTER 4 Business Intelligence (BI) Tools for Data Analysis

P A R T 1 DATA ANALYSIS

09953_ch02_ptg01_037-083.indd 37 04/03/19 10:41 PM

CHAPTER 2 Describing the Distribution of a Variable

RECENT PRESIDENTIAL ELECTIONS Presidential elections in the United States are scrutinized more than ever. It hardly seems that one is over before we start hearing plans and polls for the next. There is thor- ough coverage of the races leading up to the elections, but it is also interesting to analyze the results after the elec- tions have been held. This is not difficult, given the many informative websites that appear immediately with elec- tion results. For example, a Web search for “2016 pres- idential election results” finds many sites with in-depth results, interactive maps, and more. In addition, the resulting data can often be imported into Excel® for fur- ther analysis.

The file Presidential Elections.xlsx contains such downloaded data for the 2000 (Bush versus Gore), 2004 (Bush versus Kerry), 2008 (Obama versus McCain), 2012 (Obama versus Romney), and 2016 (Trump versus Clinton) elec- tions. The results of the 2000 election are particularly interesting. As you might remember, this was one of the closest elections of all time, with Bush defeating Gore by a very narrow margin in the electoral vote, 271 to 266, following a dis- puted recount in Florida. In fact, Gore actually beat Bush in the total count of U.S. votes, 50,999,897 to 50,456,002. However, because of the all-or-nothing nature of electoral votes in each state, Bush’s narrow margin of victory in many closely con- tested states won him a lot of electoral votes. In contrast, Gore outdistanced Bush by a wide margin in several large states, winning him the same electoral votes he would have won even if these races had been much closer.

A closer analysis of the state-by-state results shows how this actually hap- pened. The Excel file contains two new columns: Bush Votes minus Gore Votes and Pct for Bush minus Pct for Gore, with a value for each state (including the District of Columbia). We then created column charts of these two variables, as shown in Figures 2.1 and 2.2.

Each of these charts tells the same story, but in slightly different ways. From Figure 2.1, you can see how Gore won big (large vote difference) in several large states, most notably California, Massachusetts, and New York. Bush’s only comparable margin of victory was in his home state of Texas. However, Bush won a lot of close races in states with relatively few electoral votes—but enough to add up to an overall win. As Figure 2.2 indicates, many of these “close” races, such as Alaska and Idaho for Bush

S c o tt O

ls o n /G

e tt y

Im a g

e s

N e w

s/ G

e tt

y Im

a g

e s

09953_ch02_ptg01_037-083.indd 38 04/03/19 10:41 PM

2-1 Introduction 3 9

and District of Columbia for Gore, were not that close after all, at least not from a per- centage standpoint. This is one case of many where multiple charts can be created to “tell a story.” Perhaps an argument can be made that Figure 2.1 tells the story best, but Figure 2.2 is also interesting.

Figure 2.1 Chart of Vote Differences

–2,000,000

–1,500,000

–1,000,000

–500,000

500,000

1,000,000

1,500,000

2,000,000

A la

ba m

a A

la sk

a A

ri zo

na A

rk an

sa s

Ca lif

or ni

a Co

lo ra

D el

aw ar

e D

is tr

ic t o

f C ol

um bi

a Fl

or id

a G

eo rg

ia H

aw ai

i Id

ah o

Ill in

oi s

In di

an a

Io w

a Ka

ns as

Ke nt

uc ky

Lo ui

si an

a M

ai ne

M ar

yl an

M ic

hi ga

n M

in ne

so ta

M is

si ss

ip pi

M is

so ur

i M

on ta

na N

eb ra

sk a

N ev

ad a

N ew

H am

ps hi

re N

ew Je

rs ey

N ew

M ex

ic o

N ew

Y or

k N

or th

C ar

ol in

a N

or th

D ak

ot a

O hi

o O

kl ah

om a

O re

go n

Pe nn

sy lv

an ia

Rh od

e Is

la nd

So ut

h Ca

ro lin

a So

ut h

D ak

ot a

Te nn

es se

e Te

xa s

U ta

h Ve

rm on

t Vi

rg in

ia W

as hi

ng to

n W

es t V

ir gi

ni a

W is

co ns

in W

yo m

in g

Votes for Bush minus Votes for Gore

Figure 2.2 Chart of Percent Differences

–100.00%

60.00%

A la

ba m

W yo

m in

Pct for Bush minus Pct for Gore

A la

sk a

A ri

zo na

A rk

an sa

s Ca

lif or

ni a

Co lo

ra do

D el

aw ar

e D

is tr

ic t o

f C ol

um bi

a Fl

or id

a G

eo rg

ia H

aw ai

i Id

ah o

Ill in

oi s

In di

an a

Io w

a Ka

ns as

Ke nt

uc ky

Lo ui

si an

a M

ai ne

M ar

yl an

M ic

hi ga

n M

in ne

so ta

M is

si ss

ip pi

M is

so ur

i M

on ta

na N

eb ra

sk a

N ev

ad a

N ew

H am

ps hi

re N

ew Je

rs ey

N ew

M ex

ic o

N ew

Y or

k N

or th

C ar

ol in

a N

or th

D ak

ot a

O hi

o O

kl ah

om a

O re

go n

Pe nn

sy lv

an ia

Rh od

e Is

la nd

So ut

h Ca

ro lin

a So

ut h

D ak

ot a

Te nn

es se

e Te

xa s

U ta

h Ve

rm on

t Vi

rg in

ia W

as hi

ng to

n W

es t V

ir gi

ni a

W is

co ns

–80.00%

–60.00%

–40.00%

–20.00%

0.00%

20.00%

40.00%

The bottom line is that the election could easily have gone the other way. With one more swing state, particularly Florida, Al Gore would have been president. On the other hand, Gore won some very close races as well, particularly in Iowa, Minnesota, New Mexico, and Oregon. If these had gone the other way, the popular vote would still have been very close, but Bush’s victory in the electoral vote would have been more impressive.

2-1 Introduction The goal of this chapter and the next two chapters is very simple: to make sense of data by constructing appropriate summary measures, tables, and graphs. The purpose here is to present the data in a form that makes sense to people. There are numerous ways to do this, limited only by your imagination (and the software you are using), but there are

09953_ch02_ptg01_037-083.indd 39 04/03/19 10:41 PM

4 0 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

several tools used most often: (1) a variety of graphs, including bar charts, pie charts, histograms, scatterplots, and time series graphs; (2) numeric summary measures such as counts, percentages, averages, and measures of variability; and (3) tables of summary measures such as totals, averages, and counts, grouped by categories. These terms might not all be familiar to you at this point, but you have undoubtedly seen examples of them in newspapers, magazine articles, and books.

The material in these three chapters is simple, complex, and important. It is simple because there are no difficult mathematical concepts. With the possible exception of vari- ance, standard deviation, and correlation, all the numeric measures, graphs, and tables are natural and easy to understand.

If it is so easy, why do we also claim that the material in these chapters is com- plex? The data sets available to companies in today’s digital world tend to be extremely large and filled with “unstructured” data. As you will see, even in data sets that are quite small in comparison with those that real companies face, it is a challenge to summarize the data so that the important information stands out clearly. It is easy to produce summary measures, graphs, and tables, but the real goal is to produce the most insightful ones.

The typical employees of today—not just the managers and technical special- ists—have a wealth of software tools at their disposal, and it is frequently up to them to summarize data in a way that is both meaningful and useful to their constituents: people within their company, their company’s stockholders, their company’s suppli- ers, and their company’s customers. It takes some training and practice to do this effectively.

Data analysis in the real world is almost never done in a vacuum. It is almost always done to solve a problem. Typically, there are four steps that are followed, whether the context is business, medical science, or any other field. The first step is to recognize a problem that needs to be solved. Perhaps a retail company is experiencing decreased sales in a particular region or for a particular product. Why is this happening? The sec- ond step is to gather data to help understand and then solve the problem. This might be done through a survey of customers, by assembling data from already-existing company systems, by finding relevant data on the Web, or from other sources. Once the data are gathered, the third step is to analyze the data using the tools you will learn in the book. The fourth step is to act on this analysis by changing policies, undertaking initiatives, publishing reports, and so on. Of course, the analysis can sometimes repeat steps. For example, once a given set of data is analyzed, it might be apparent that even more data needs to be collected.

As we discuss the tools for analyzing data, we will often jump to the third step directly, providing you with a data set to analyze. Although this data set may not be directly connected to the goal of solving a company’s problem, you should still strive to ask interesting questions of the data. (We have tried to include interesting data sets, often containing real data, that make this possible.) If the data set contains salaries, you might ask what drives these salaries. Does it depend on the industry a person is in? Does it depend on gender? Does it depend on educational background? Is the salary structure, whatever it is, changing over time? If the data set contains cost-of-living indexes, there are also a lot of interesting questions you can ask. How are the indexes changing over time? Does this behavior vary in different geographical regions? Does this behavior vary across different items such as housing, food, and automobiles? These early chapters provide you with many tools to answer such questions, but it is up to you to ask good questions—and then take advantage of the most appropriate tools to answer them.

The material in these chapters is organized as follows. This chapter discusses a number of ways to analyze one variable at a time. Chapter 3 discusses ways to discover relation- ships between variables. Finally, Chapter 4, a new chapter in this edition of the book, takes data analysis to a higher level. It first discusses two relatively new tools in Excel, Power

It is customary to refer to the raw numbers as data and the output of data anal- ysis as information. You start with the data, and your goal is to end with information that an organization can use for competitive advantage.

Use your imagination to ask interesting questions about the many data sets available to you. This book will supply you with the tools to answer these questions.

09953_ch02_ptg01_037-083.indd 40 04/03/19 10:41 PM

2-2 Basic Concepts 4 1

Query and Power Pivot, for importing and analyzing data. It then discusses Tableau Public, a free non-Microsoft software package for creating insightful visualizations of data.

2-2 Basic Concepts We begin with a short discussion of several important concepts: populations and samples, data sets, variables and observations, and types of data.

2-2a Populations and Samples First, we distinguish between a population and a sample. A population includes all of the entities of interest: people, households, machines, or whatever. The following are three typical populations:

• All potential voters in a presidential election • All subscribers to a cable television provider • All invoices submitted for Medicare reimbursement by nursing homes

In these situations and many others, it is virtually impossible to obtain information about all members of the population. For example, it is far too costly to ask all potential voters which presidential candidates they prefer. Therefore, we often try to gain insights into the characteristics of a population by examining a sample, or subset, of the population. In later chapters, we examine populations and samples in some depth, but for now it is enough to know that samples should be representative of the population so that observed characteristics of the sample can be generalized to the population as a whole.

A population includes all of the entities of interest in a study. A sample is a subset of the population, often randomly chosen and preferably representative of the population as a whole.

We use the terms population and sample a few times in this chapter, which is why we have defined them here. However, the distinction is not really important until later chap- ters. Our intent in this chapter is to focus entirely on the data in a given data set, not to generalize beyond it. Therefore, the given data set could be a population or a sample from a population. For now, the distinction is irrelevant. (Sampling is discussed in more detail in Chapter 7.)

2-2b Data Sets, Variables, and Observations We now discuss the types of data sets we will examine. Although the focus of this book is Excel, virtually all statistical software packages use the same concept of a data set: A data set is generally a rectangular table of data where the columns contain variables, such as height, gender, and income, and each row contains an observation. Each observation includes the attributes of a particular member of the population: a person, a company, a city, a machine, or whatever. This terminology is common, but other terms are often used. A variable (column) is often called a field or an attribute, and an observation (row) is often called a case or a record. Also, data sets are occasionally rearranged, so that the variables are in rows and the observations are in columns. However, the most common arrangement by far is to have variables in columns, with variable names in the top row, and observations in the remaining rows.

09953_ch02_ptg01_037-083.indd 41 04/03/19 10:41 PM

4 2 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

In anticipation of Chapter 4, this definition of “data set” as a single table of data is often expanded to several related tables of data. However, for this chapter and the next chapter, each data set will be restricted to a single table of data.

A data set is usually a rectangular table of data, with variables in columns and observations in rows. A variable (or field or attribute) is a characteristic of members of a population, such as height, gender, or salary. An observation (or case or record) is a list of all variable values for a single member of a population.

EXAMPLE

2.1 DATA FROM AN ENVIRONMENTAL SURVEY The data set in Figure 2.3 shows a few of the 30 responses from a questionnaire concerning the president’s environmental policies. (See the file Questionnaire Data.xlsx.)

1 2 3 4 5 6 7 8 9

A B C D E F G Person Age Gender State Children Salary

1 35 Male Minnesota 1 $65,400 2 61 Female Texas 2 $62,000 3 35 Male Ohio 0 $63,200 4 37 Male Florida 2 $52,000 5 32 Female California 3 $81,400 6 33 Female

Female New York 3 $46,300

7 65 Minnesota 2 $49,600 8 45 New York $45,900

10 11

Male 1 9 40 Male Texas 3 $47,700

10 32 Female Texas 1 $59,900

Opinion 5 1 3 5 1 5 1 5 4 4

Figure 2.3 Environmental Survey Data

Objective To illustrate variables and observations in a typical data set.

Solution Each observation lists the person’s age, gender, state of residence, number of children, annual salary, and opinion of the president’s environmental policies. These six pieces of information represent the variables. It is customary to include a row (row 1 in this case) that lists variable names. These variable names should be concise but meaningful. Note that an index of the observation is often included in column A. If you sort on other variables, you can always sort on the index to get back to the original sort order.

2-2c Data Types There are several ways to categorize data, as we explain in the context of Example 2.1. A basic distinction is between numeric and categorical data. The distinction is whether you intend to do any arithmetic on the data. It makes sense to do arithmetic on numeric data, but not on categorical data. Actually, there is a third data type, a date variable. As you may know, Excel stores dates as numbers, but for obvious reasons, dates are treated differently from typical numbers.

Three types of variables that appear to be numeric but are usually treated as categorical are phone numbers, zip codes, and Social Security numbers. Do you see why? Can you think of others?

A variable is numeric if meaningful arithmetic can be performed on it. Otherwise, the variable is categorical.

09953_ch02_ptg01_037-083.indd 42 04/03/19 10:41 PM

2-2 Basic Concepts 4 3

In the questionnaire data set, Age, Children, and Salary are clearly numeric. For example, it makes perfect sense to sum or average any of these. In contrast, Gender and State are clearly categorical because they are expressed as text, not numbers.

The Opinion variable is less obvious. It is expressed numerically, on a 1-to-5 scale. However, these numbers are really only codes for the categories “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” There is never any intent to perform arithmetic on these numbers; in fact, it is not really appropriate to do so. Therefore, it is most appropriate to treat the Opinion variable as categorical. Note, too, that there is a defi- nite ordering of its categories, whereas there is no natural ordering of the categories for the Gender or State variables. When there is a natural ordering of categories, the variable is classified as ordinal. If there is no natural ordering, as with the Gender or State variables, the variable is classified as nominal.

A categorical variable is ordinal if there is a natural ordering of its possible categories. If there is no natural ordering, the variable is nominal.

Categorical variables can be coded numerically. In Figure 2.3, Gender has not been coded, whereas Opinion has been coded. This is largely a matter of taste—so long as you realize that coding a categorical variable does not make it numeric and appropriate for arithmetic operations. An alternative way of displaying the data set appears in Figure 2.4. Now Opinion has been replaced by text, and Gender has been coded as 1 for males and 0 for females. This 021 coding for a categorical variable is very common. Such a variable is usually called a dummy variable, and it often simplifies the analysis. You will see dummy variables throughout the book.

Horizontal Alignment Conventions

Excel automatically right-aligns numbers and left-aligns text. We use this automatic formatting, but we add our own. Specifically, we right-align all numbers that are available for arithmetic; we left-align all text such as Male, Female, Yes, and No; and we center-align everything else, including dates, indexes such as the Person column, numbers that are indicators of categories such as in the Opinion column, and numbers such as phone numbers that aren’t available for arithmetic. You don’t need to follow this convention, but it helps to identify the appropriate roles of variables.

Excel Tip

Documenting with Cell Comments

To remember, for example, that “1” stands for “strongly disagree” in the Opinion variable, you can enter a comment—a reminder to yourself and others—in any cell. To do so, right-click a cell and select Insert Comment. A small red tag appears in any cell with a comment. Moving the cursor over that cell causes the comment to appear. You will see numerous comments in the files that accompany the book.

Excel Tip

09953_ch02_ptg01_037-083.indd 43 04/03/19 10:41 PM

4 4 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

In addition, the Age variable has been categorized as “young” (34 years or younger), “middle-aged” (from 35 to 59 years), and “elderly” (60 years or older). This method of cat- egorizing a numeric variable is called binning (putting the data into discrete bins), and it is also common. (It is also called discretizing.) The purpose of the study dictates whether age should be treated numerically or categorically; there is no absolute right or wrong way.

A dummy variable is a 021 coded variable for a specific category. It is coded as 1 for all observations in that category and 0 for all observations not in that category.

1 2 3 4 5 6 7 8 9

A B C D E F G H I J K L Person Age Gender State Children Salary Opinion

1 1 Minnesota 1 2 Elderly 0 Texas 2 3 1 Ohio 0 Neutral 4 1 Florida 2 5 Young 0 California 3 6 Young 0 3 Strongly agree

Strongly agree

7 Elderly 0 Minnesota 2 8 1 New York

New York

1 Strongly agree

Strongly disagree

Strongly disagree Note the formulas in columns B, C, and G that generate this recoded data. The formulas in columns B and G are based on the lookup tables below.

9 10 11 12 13 14 15 16 17 18 19

1 9 1 Texas 3 $47,700

$49,600 $46,300 $81,400 $52,000 $63,200 $62,000 $65,400

$45,900 Agree

10 Young 0 Texas 1 $59,900 Agree 11 1 New York 1 $48,100 Agree 12 0 Virginia 0 $58,100 Neutral 13 0 Illinois 2 $56,000 14 Middle-aged

Middle-aged Middle-aged Middle-aged

Middle-aged Middle-aged

Middle-aged

0 Virginia 2 $53,400 Strongly disagree Strongly disagree

Strongly disagree

Strongly agree

0 Young

15 Middle-aged 0 New York 2 $39,000

Middle-aged53

16 Middle-aged 1 Michigan 1 $61,500

Elderly06

Disagree Disagree

17 Middle-aged 1 Ohio 0 $37,700 Strongly disagree 18 Middle-aged 0 Michigan 219

28 29

18 2 $36,700 Agree

1 Disagree2 Neutral3 Agree4

29 30 31

27 Young 1 Illinois 3 $45,400 Disagree 28 1 2 $53,90028 Elderly Michigan 2 Strongly disagree 29 Middle-aged 1 California 1 $44,100 Neutral 30 Middle-aged 0 New York 2 $31,000 Agree

Opinion lookup table (range name OpinionLookup)

Age lookup table (range name AgeLookup)

Figure 2.4 Environmental Data Using a Different Coding

A binned (or discretized) variable corresponds to a numeric variable that has been categorized into discrete categories. These categories are usually called bins.

VLOOKUP Function

As Figure 2.4 indicates, we used lookup tables, along with the very important VLOOKUP function, to transform the data set from Figure 2.3 to Figure 2.4. Take a look at these functions in the questionnaire file. There is arguably no more important Excel function than VLOOKUP, so you should definitely learn how to use it. If you need help, consult the Excel Tutorial files that accompany the book.

Excel Tip

Numeric variables can be classified as discrete or continuous. The basic distinction is whether the data arise from counts or continuous measurements. The variable Children is clearly a count (discrete), whereas the variable Salary is best treated as continuous. This distinction between discrete and continuous variables is often important because it dictates the most natural type of analysis.

09953_ch02_ptg01_037-083.indd 44 04/03/19 10:41 PM

2-3 Summarizing Categorical Variables 4 5

A numeric variable is discrete if it results from a count, such as the number of children. A continuous variable is the result of an essentially continuous measurement, such as weight or height.

Data sets can also be categorized as cross-sectional or time series. The opinion data set in Example 2.1 is cross-sectional. A pollster evidently sampled a cross section of peo- ple at one particular point in time. In contrast, a time series data set tracks one or more variables through time. A typical time series variable is the series of daily closing values of the Dow Jones Industrial Average (DJIA). Very different types of analyses are appropriate for cross-sectional and time series data, as will become apparent in this and later chapters.

Cross-sectional data are data on a cross section of a population at a distinct point in time. Time series data are data collected over time.

A time series data set generally has the same layout—variables in columns and observations in rows—but now each variable is a time series. Also, one of the columns usually indicates the time period. A typical example appears in Figure 2.5. (See the file Toy Revenues.xlsx.) It has quarterly observations on revenues from toy sales over a four- year period in column B, with the time periods listed chronologically in column A.

1 2 3 4 5 6

A B C Quarter Revenue Q1-2015 $1,026,000 Q2-2015 $1,056,000 Q3-2015 $1,182,000 Q4-2015 $2,861,000 Q1 2016 $1,172,000

7 8 9

10 11 12

- Q2-2016 $1,249,000 Q3-2016 $1,346,000 Q4-2016 $3,402,000 Q1-2017 $1,286,000 Q2-2017 $1,317,000 Q3-2017 $1,449,000

13 14 15 16 17

Q4-2017 $3,893,000 Q1-2018 $1,462,000 Q2-2018 $1,452,000 Q3-2018 $1,631,000 Q4-2018 $4,200,000

Figure 2.5 Typical Time Series Data Set

2-3 Summarizing Categorical Variables This section discusses methods for summarizing a categorical variable. Because it is not appropriate to perform arithmetic on the values of the variable, there are only a few pos- sibilities for summarizing the variable, and these are all based on counting. First, you can count the number of categories. Many categorical variables such as Gender have only two categories. Others such as Region can have more than two categories. As you count the categories, you can also give the categories names, such as Male and Female. Keep in

09953_ch02_ptg01_037-083.indd 45 04/03/19 10:41 PM

4 6 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

mind that categorical variables, such as Opinion in Example 2.1, can be coded numeri- cally. In these cases, it is still a good idea to supply text descriptions of the categories, such as “strongly agree,” and it is often useful to substitute these meaningful descriptions for the numeric codes, as in Figure 2.4. This is especially useful for statistical reports.

Once you know the number of categories and their names, you can count the number of observations in each category. The resulting counts can be reported as “raw counts” or as percentages of totals. For example, if there are 1000 observations, you can report that there are 560 males and 440 females, or you can report that 56% of the observations are males and 44% are females. Actually, it is often useful to report the counts in both ways. Finally, once you have the counts, you can display them graphically, usually in a column chart or possibly a pie chart. The following example illustrates how to do this in Excel.

The only meaningful way to summarize categorical data is with counts of observations in the categories.

EXAMPLE

2.2 SUPERMARKET SALES The file Supermarket Transactions.xlsx contains over 14,000 transactions made by supermarket customers over a period of approximately two years. (The data are not real, but real supermarket chains have huge data sets similar to this one.) A small sample of the data appears in Figure 2.6. Column B contains the date of the purchase, column C is a unique identifier for each customer, columns D–H contain information about the customer, columns I–K contain the location of the store, columns L–N (hidden to conserve space) contain information about the product purchased, and the last two columns indicate the number of items purchased and the amount paid.

1 2 3 4 5 6 7 8 9

10 11

A B C D E F G H I J K O P Purchase DateTransaction

Customer ID Gender

Marital Status Homeowner Children Annual Incom e City

State or Provinc e Country

Units Sold Revenue

1 12/18/2016 7223 F S Y 2 $30K - $50K Los Angele s CA USA 5 $27.38 2 12/20/2016 7841 M M Y 5 $70K - $90K Los Angele s CA USA 5 $14.90 3 12/21/2016 8374 F M N 2 $50K - $70K Bremerto n WA USA 3 $5.52 4 12/21/2016 9619 M M Y 3 $30K - $50K Portlan d OR USA 4 $4.44 5 12/22/2016 1900 F S Y 3 $130K - $150K Beverly Hills CA USA 4 $14.00 6 12/22/2016 6696 F M Y 3 $10K - $30K Beverly Hills CA USA 3 $4.37 7 12/23/2016 9673 M S Y 2 $30K - $50K Salem OR USA 4 $13.78 8 12/25/2016 354 F M Y 2 $150K + Yakima WA USA 6 $7.34 9 12/25/2016 1293 M M Y 3 $10K - $30K Bellingha m WA USA 1 $2.41

10 12/25/2016 7938 M S N 1 $50K - $70K San Diego CA USA 2 $8.96

Figure 2.6 Supermarket Data Set

Objective To summarize categorical variables in a large data set.

Solution Most of the variables in this data set are categorical. Only Children, Units Sold, and Revenue are numeric. Purchase Date is a date variable, and Transaction and Customer ID are used only to identify transactions and customers. All other variables are categorical, including Annual Income, which has been binned into categories. Three of the categorical variables—Gender, Marital Status, and Homeowner—have only two categories. The others have more than two categories.

The first question is how you can discover all of the categories for a variable such as State or Province. Without good tools, this is not a trivial problem. One option is to sort on this variable and then manually go through the list, looking for the different categories. Another possibility is to copy the categorical column to a blank area of the worksheet and then use the Remove Duplicates item on Excel’s Data ribbon. (Try it!) Fortunately, there are easier ways, using Excel’s built-in table and pivot table tools. We postpone these for later and deal for now only with the categorical variables with obvious categories.

Figure 2.7 displays summaries of Gender, Marital Status, Homeowner, and Annual Income, along with several corre- sponding charts for Gender. Each of the counts in column S can be obtained with Excel’s COUNTIF function. For example, the formula in cell S3 is 5COUNTIF($D$2:$D$14060,R3). This function takes two arguments, the data range and a criterion, so it is perfect for counting observations in a category. Then the percentages in column T are the counts divided by the total number of observations. (As a check, it is a good idea to sum these percentages. They should sum to 100% for each variable.)

09953_ch02_ptg01_037-083.indd 46 04/03/19 10:41 PM

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

R S T U V W X Y Z AA AB AC AD AE Categorical summaries Gender Count Percent M 6889 49.0% F 7170 51.0%

100.0%

Marital Status Count Percent S 7193 51.2% M 6866 48.8%

100.0%

Homeowner Count Percent Y 8444 60.1% N 5615 39.9%

100.0%

Annual Income Count Percent $10K - $30K 3090 22.0% $30K - $50K 4601 32.7% $50K - $70K 2370 16.9% $70K - $90K 1709 12.2% $90K - $110K 613 4.4% $110K - $130K 643 4.6% $130K - $150K 760 5.4% $150K + 273 1.9%

100.0%

6700 6800 6900 7000 7100 7200

M F

Gender Count

0 2000 4000 6000 8000

M F

Gender Count (different scale)

48.0% 49.0% 50.0% 51.0% 52.0%

M F

Gender Percent

0.0%

20.0%

40.0%

60.0%

M F

Gender Count

Gender Percent

Gender Percent (different scale)

Figure 2.7 Summaries of Categorical Variables

As the charts indicate, you get essentially the same chart whether you graph the counts or the percentages. However, be careful with misleading scales. If you select the range R2:S4 and then insert a column chart, you get the top left chart by default. Its vertical scale starts well above 6000, which makes it appear that there are many more females than males. By resetting the vertical scale to start at 0, as in the two middle charts, you see more accurately that there are almost as many males as females. Finally, you can decide whether you prefer a column chart or a pie chart. We tend to prefer column charts, but this is a matter of taste. (We also tend to prefer column charts to horizontal bar charts, but this is also a matter of taste.) Our only recommendation in general is to keep charts simple so that the information they contain emerges as clearly as possible.

When you have a choice between a “simple” chart and a “fancier” chart, keep it simple. Simple charts tend to reveal the information in the data more clearly.

If this example of summarizing categorical variables appears to be overly tedious, be patient. As indicated earlier, Excel has powerful tools, especially pivot tables, that make this summarization much easier. We discuss pivot tables in depth in the next chapter. For now, just remember that the only meaningful way to summarize a categorical variable is to count observa- tions in its categories.

Creating and Modifying an Excel Chart

If you are new to Excel charts, you should try creating the charts in Figure 2.7 on your own. One way is to select a blank cell, select a desired chart type from the Insert ribbon, and then designate the data to be included in the chart. However, it is usually simpler and more efficient to select the data to be charted and then insert the chart. For example, try highlighting the range R2:S4 and then inserting a column chart. Except for a little cleanup (changing the chart title, and possibly changing the vertical scale), you get almost exactly what you want with very little work. Don’t be afraid to right-click for context-sensitive chart menus or to use the special ribbons that appear when a chart is selected. You can learn a lot by experimenting with these tools.

Excel Tip

2-3 Summarizing Categorical Variables 4 7

09953_ch02_ptg01_037-083.indd 47 04/03/19 10:41 PM

Before leaving this section, we mention one other efficient way to find counts for a categorical variable. This method uses dummy (021) variables. To see how it works, focus on any category of a categorical variable, such as M for Gender. Recode the variable so that each M is replaced by a 1 and all other values are replaced by 0. (This can be done in Excel in a new column, using a simple IF formula. See column E of Figure 2.8.) Now you can find the count of males by summing the 0’s and 1’s, and you can find the percentage of males by averaging the 0’s and 1’s. That is, the formulas in cells E14061 and E14062 use the SUM and AVERAGE functions on the data in column E. You should convince yourself why this works (for example, what arithmetic are you really doing when you average 0’s and 1’s?), and you should remember this method. It is one reason why dummy variables are used so frequently in data analysis.

1 2 3 4 5 6 7

1 2 3 4 5 6

14057 14058 14059

14058 14059

14061 14062

14060

A B C D E Purchase Date Customer ID Gender Gender Dummy for MTransaction

F M F M F F

12/18/2016 12/20/2016 12/21/2016 12/21/2016 12/22/2016 12/22/2016 12/31/2018 12/31/2018 12/31/2018

7223 7841 8374 9619 1900 6696 250

3656 6153

M F

Count Percent

0 1 0 1 0 0 1 0 1

6889 49.0%

Figure 2.8 Summarizing a Category with a Dummy Variable

4 8 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. The file P02_01.xlsx indicates the gender and national-

ity of the MBA incoming class in two successive years at the Kelley School of Business at Indiana University. a. For each year, create tables of counts of gender and

of nationality. Then create column charts of these counts. Do they indicate any noticeable change in the composition of the two classes?

b. Repeat part a for nationality, but recode this variable so that all nationalities that have counts of 1 or 2 are classified as Other.

2. The file P02_02.xlsx contains information on 256 movies that grossed at least $1 million in 2017. a. Create two column charts of counts, one of the differ-

ent genres and one of the different distributors. b. Recode the Genre column so that all genres with a

count of 10 or less are lumped into a category called Other. Then create a column chart of counts for this recoded variable. Repeat similarly for the Distributor variable.

3. The file P02_03.xlsx contains data from a survey of 399 people regarding a government environmental policy. a. Which of the variables in this data set are categorical?

Which of these are nominal; which are ordinal? b. For each categorical variable, create a column chart of

counts.

09953_ch02_ptg01_037-083.indd 48 04/03/19 10:41 PM

2-4 Summarizing Numeric Variables 4 9

c. Recode the data into a new data set, making four transformations: (1) change Gender to list “Male” or “Female”; (2) change Children to list “No children” or “At least one child”; (3) change Salary to be cate- gorical with categories “Less than $40K,” “Between $40K and $70K,” “Between $70K and $100K,” and “Greater than $100K ” (where you can treat the break- points however you like); and (4) change Opinion to be a numeric code from 1 to 5 for Strongly Disagree to Strongly Agree. Then create a column chart of counts for the new Salary variable.

4. The file P02_04.xlsx contains salary data on all Major League Baseball players for each year from 2012 to 2018. For any three selected years, create a table of counts of the various positions, expressed as percentages of all players for the year. Then create a column chart of

these percentages for these years. Do they remain fairly constant from year to year?

Level B 5. The file P02_05.xlsx contains monthly values of the

Dow Jones Industrial Average from 1950 until recently. It also contains the percentage changes from month to month. (This file is used in an example later in this chapter.) Create a new column for recoding the percent- age changes into six categories: Large negative ( 623%), Medium negative ( 621%, G23%), Small negative ( 60%, G21%), Small positive ( 61%, G 0%), Medium positive ( 63%, G1%), and Large positive ( G 3%). Then create a column chart of the counts of this categorical variable. Comment on its shape.

2-4 Summarizing Numeric Variables There are many ways to summarize numeric variables, both with numeric summary measures and with charts, and we discuss the most common ways in this section. But before getting into details, it is important to understand the basic goal of this section. Starting with a numeric variable such as Salary, where there is one observation for each person, the basic goal is to learn how these salaries are distributed across people. To do this, we can ask a number of questions, including the following: (1) What are the most “typical” salaries? (2) How spread out are the salaries? (3) What are the “extreme” salaries on either end? (4) Is a chart of the salaries symmetric about some middle value, or is it skewed in one direction? (5) Does the chart of salaries have any other peculiar features besides possible skewness? In the next chapter, we explore methods for checking whether a variable such as Salary is related to other variables, but for now the goal is simply to explore the distribution of values in the Salary column.

2-4a Numeric Summary Measures Throughout this section, we focus on a Salary variable. Specifically, we examine the 2018 salaries for Major League Baseball players, as described in the following extended example.

Statistical Functions

Microsoft introduced a number of new or modified statistical functions in Excel 2010, but for backward compatibility, it also retained the older versions. You can recognize the new functions with periods, such as NORM.DIST (new) versus NORMDIST (old). Because these new functions have now existed for nearly a decade, we usually use them in this book, but if you are used to the older versions, you can continue to use them.

Excel Tip

09953_ch02_ptg01_037-083.indd 49 04/03/19 10:41 PM

5 0 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

EXAMPLE

2.3 BASEBALL SALARIES The file Baseball Salaries.xlsx contains data on 877 Major League Baseball (MLB) players in the 2018 season. (The file also contains similar data for each of the years 2012 to 2017.) There are four variables, as shown in Figure 2.9: the player’s name, team, position, and salary. How can these 877 salaries be summarized?

Objective To learn how salaries are distributed across all 2018 MLB players.

Solution The various numeric summary measures can be categorized into several groups: measures of central tendency; minimum, maxi- mum, percentiles, and quartiles; measures of variability; and measures of shape. We explain each of these in this extended example.

Measures of Central Tendency There are three common measures of central tendency, all of which try to answer the basic question of which value is most “typical.” These are the mean, the median, and the mode.

The mean is the average of all values. If the data set represents a sample from some larger population, this measure is called the sample mean and is usually denoted by X (pronounced “X-bar”). If the data set represents the entire population, it is called the population mean and is usually denoted by m (the Greek letter mu). This distinction is not important in this chapter, but it will become relevant in later chapters on statistical inference. In either case, the formula for the mean is given by Equation (2.1).

1 2 3 4 5 6 7 8 9

10 11 12 13 14

DCBA SalaryPositionTeamName

Clayton Kershaw Mike Trout

Zack Greinke Miguel Cabrera David Price Jake Arrieta Yoenis Cespedes Justin Verlander Jon Lester Albert Pujols Felix Hernandez Jason Heyward Giancarlo Stanton

Los Angeles Dodgers

Detroit Tigers Arizona Diamondbacks

Boston Red Sox Philadelphia Phillies New York Mets Houston Astros Chicago Cubs

Los Angeles Angels

Los Angeles Angels Seattle Mariners Chicago Cubs New York Yankees

Outfielder Pitcher Pitcher First Baseman Pitcher Pitcher Outfielder Pitcher Pitcher First Baseman Pitcher Outfielder Outfielder

$34,083,333 $34,000,000 $31,954,483 $30,000,000 $30,000,000 $30,000,000 $29,000,000 $28,000,000 $27,500,000 $27,000,000 $26,857,143 $26,055,288 $25,000,000

Figure 2.9 Baseball Salaries

Formula for the Mean

Mean 5 an i 5 1

n (2.1)

Here, n is the number of observations and Xi is the value of observation i. Equation (2.1) says to add all the observations and divide by n, the number of observations. The S (Greek capital sigma) symbol means to sum from i 5 1 to i 5 n, that is, to sum over all observations.

For Excel data sets, you can calculate the mean with the AVERAGE function. This is shown for the baseball data, along with many other summary measures discussed shortly, in Figure 2.10. Specifically, the average salary for all players is $4,512,768. Is this a “typical” salary? Keep reading.

09953_ch02_ptg01_037-083.indd 50 04/03/19 10:41 PM

In many situations like this, where the data are skewed to the right (a few extremely large salaries not balanced by any extremely small salaries), most people would argue that the median is a more representative measure of central tendency than the mean. However, for variables that are not skewed in one direction or the other, the mean and median are often quite close to one another.

The mode is the value that appears most often, and it can be calculated in Excel with the MODE function. In most cases where a variable is essentially continuous, the mode is not very interesting because it is often the result of a few lucky ties. However, the mode for the salary data in Figure 2.10 is not the result of luck. Its value, $545,000, is the minimum possible salary set by the league. As shown in cell C4 (with a COUNTIF formula), this value occurred 44 times. In other words, close to 5% of the players earn the minimum possible salary. This is a good example of learning something you might not have known simply by exploring the data.

Minimum, Maximum, Percentiles, and Quartiles Given a certain percentage such as 25%, what is the salary value such that this percentage of salaries is below it? This type of question leads to percentiles and quartiles. Specifically, for any percentage p, the pth percentile is the value such that

The median is the middle observation when the data are sorted from smallest to largest. If the number of observations is odd, the median is literally the middle obser- vation. For example, if there are nine observations, the median is the fifth smallest (or fifth largest). If the number of observations is even, the median is usually defined as the average of the two middle observations, although there are some slight varia- tions of this definition. For example, if there are 10 observations, the median is usually defined as the average of the fifth and sixth smallest values.

The median can be calculated in Excel with the MEDIAN function. Figure 2.10 shows that the median salary is $1,450,000. In words, half of the players make less than this, and half make more. Why is the median in this example so much smaller than the mean, and which is more appropriate? These are relevant questions for many real-world data sets. In this case, the vast majority of baseball players have relatively modest salaries that are dwarfed by the huge salaries of a few stars. Because it is an average, the mean is strongly influenced by these really large values, so it is quite large. In contrast, the median is completely unaffected by the magnitude of the really large salaries, so it is much smaller. (For example, the median would not change by a single cent if the highest paid player made $34 billion instead of his $34 million, but the mean would increase to more than $43 million.)

For highly skewed data, the median is typically a better measure of central tendency. The median is unaffected by the extreme values, whereas the mean can be very sensitive to extreme values.

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20

FEDCBA Measures of central tendency Measures of variability

$33,538,333 $5,442,529

38,845,755,393,289 $6,232,636 $4,553,478

Range Interquartile range Variance Standard deviation Mean absolute deviation

$4,512,768Mean $1,450,000 Mode countMedian

Mode $545,000 44

Min, max, percentiles, quartiles Min $545,000

$34,083,333 $557,471

$1,450,000 $6,000,000

$545,000 $545,320 $547,560 $554,640

$1,450,000 $7,733,333

$13,050,000 $19,520,000 $27,120,000

Max Measures of shape 2.0663 4.1060

Skewness Kurtosis

1 2 3

0.01 0.05 0.10 0.20 0.50 0.80 0.90 0.95 0.99

Q1 Q2 Q3 P01 Percentages of values <= given values

ValueP05 Percentage <= 45.38% 51.20% 55.53% 59.29% 62.94%

$1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000

P10 P20 P50 P80 P90 P95 P99

Figure 2.10 Summary Measures of Baseball Salaries

2-4 Summarizing Numeric Variables 5 1

09953_ch02_ptg01_037-083.indd 51 04/03/19 10:41 PM

a percentage p of all values are less than or equal to it. Similarly, the first, second, and third quartiles are the percentiles corresponding to p 5 25%, p 5 50%, and p 5 75%. These three values divide the data into four groups, each with (approxi- mately) a quarter of all observations. Note that the second quartile is equal to the median by definition. To complete this group of descriptive measures, we add the minimum and maximum values, with the obvious meanings.

You are probably aware of percentiles from standardized tests. For example, if you learn that your score in the verbal SAT test is at the 93rd percentile, this means that you scored better than 93% of those taking the test.

The minimum and maximum can be calculated with Excel’s MIN and MAX functions. For the percentiles and quartiles, you can use Excel’s PERCENTILE and QUARTILE functions. The PERCENTILE function takes two arguments: the data range and a value of p between 0 and 1. (It has to be between 0 and 1. For example, if you want the 95th percentile, you must enter the second argument as 0.95, not as 95.) The QUARTILE function also takes two arguments: the data range and 1, 2, or 3, depending on which quartile you want. Figure 2.10 shows the minimum, maximum, the three quartiles, and several commonly requested percentiles for the baseball data. As you can see, 25% of the players make no more than $557,471 (very close to the league minimum) and 25% of all players make more than $6,000,000. In fact, more than 1% of the players make more than $27 million, with Mike Trout topping the list at about $34 million.

Entering Arguments for Copying

Note the values in column C of Figure 2.10 for percentiles and quartiles. These allow you to enter one formula for the percentiles and one for quartiles that can then be copied down. Specifically, the formulas in cells B12 and B9 are

=PERCENTILE(‘Salaries 2018’!$D$2:$D$878,C12)

and

= QUARTILE(‘Salaries 2018’!$D$2:$D$878,C9)

(Here, ‘Salaries 2018’! is a reference to the worksheet that contains the data.) Always look for ways to make your Excel formulas copyable. It saves time and it avoids errors. And if you don’t want the values in column C to be visible, just color them white.

Excel Tip

Minimum and Maximum as Quartiles

Following up on the previous tip, the minimum and maximum can be calculated with Excel’s QUARTILE function by using 0 and 4 as the last argument. This means that if you enter 0 and 4 in cells C7 and C8 of Figure 2.10, the QUARTILE formula in cell B9 can be copied to cells B7 and B8.

Excel Tip

New PERCENTILE and QUARTILE Functions

Microsoft introduced four new functions, PERCENTILE.EXC, PERCENTILE.INC, QUARTILE.EXC, and QUARTILE.INC, in Excel 2010. The INC (inclusive) versions work just like the “old” PERCENTILE and QUARTILE functions. The EXC (exclusive) versions are recommended for small data sets. However, the differences are minor, and the “old” PERCENTILE and QUARTILE functions work fine.

Excel Tip

5 2 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

09953_ch02_ptg01_037-083.indd 52 04/03/19 10:41 PM

Measures of Variability In this subsection, we list a few other measures that summarize variability, that is, how much the data values are spread out. These include the range, the interquartile range, the variance and standard deviation, and the mean absolute deviation.

The range is a crude measure of variability. It is defined as the maximum value minus the minimum value. For the base- ball salaries, this range is $33,538,333. This value certainly indicates how spread out the salaries are, but it is too sensitive to the extremes. For example, if Mike Trout’s salary increased by $10 million, the range would increase by $10 million—just because of one player.

A less sensitive measure is the interquartile range (abbreviated IQR). It is defined as the third quartile minus the first quartile, so it is really the range of the middle 50% of the data. For the baseball data, the IQR is $5,442,529. If you excluded the 25% of players with the lowest salaries and the 25% with the highest salaries, this IQR would be the range of the remaining salaries.

The range or a modified range such as the IQR probably seems like a natural measure of variability, but there is another measure that is quoted much more frequently: standard deviation. Actually, there are two related measures, vari- ance and standard deviation, and we begin with a definition of variance. The variance is essentially the average of the squared deviations from the mean, where if Xi is a typical observation, its squared deviation from the mean is (Xi 2 mean)

2. As in the discussion of the mean, there is a sample variance, usually denoted by s2, and a population variance, usually denoted by s2 (where s is the Greek letter sigma). They are defined as follows:

If you are given a salary figure such as $1 million, you might want to find the percentage of all salaries less than or equal to this. This is essentially the opposite of a percentile question. In a percentile question, you are given a percentage and you want to find a value. Now you are given a value and you want to find a percentage. You can find this percentage in Excel by dividing a COUNTIF function by the total number of observations. A few such values are shown in the bottom right of Figure 2.10. The typical formula in cell F14, which is then copied down, is

5COUNTIF(‘Salaries 2018’!$D$2:$D$878,”<=”&E14)/COUNT(‘Salaries 2018’!$D$2:$D$878)

The following Excel tip explains this formula in more detail.

Creating a Condition with Concatenation

The condition in this COUNTIF formula is a bit tricky. You literally want it to be “,51000000”, but you want the formula to refer to the value in column E to enable copying. Therefore, you can concatenate (string together) the literal part, “,5”, and the variable part, the reference to cell E14. The ampersand symbol (&) in the middle is the symbol used to concatenate in Excel. This use of concatenation to join literal and variable parts is especially useful in functions like COUNTIF that require a condition, so you should learn how to use it.

Excel Tip

2-4 Summarizing Numeric Variables 5 3

Formula for Sample Variance

s2 5 an i 5 1

1Xi 2 mean22

n 2 1 (2.2)

Formula for Population Variance

s2 5 an i 5 1

1Xi 2 mean22

n (2.3)

09953_ch02_ptg01_037-083.indd 53 04/03/19 10:41 PM

To understand why the variance is indeed a measure of variability, look closely at either formula. If all observations are close to the mean, their squared deviations from the mean will be relatively small, and the variance will be relatively small. On the other hand, if at least a few of the observations are far from the mean, their squared deviations from the mean will be large, and this will cause the variance to be large. Note that because deviations from the mean are squared, an observation a certain amount below the mean contributes the same to variance as an observation that same amount above the mean.

The fundamental problem with variance as a measure of variability is that it is in squared units. For example, if the obser- vations are measured in dollars, the variance is in squared dollars. A more natural measure is the square root of variance. This is called the standard deviation. Again, there are two versions of standard deviation. The sample standard deviation, usually denoted by s, is the square root of the quantity in Equation (2.2). The population standard deviation, usually denoted by s, is the square root of the quantity in Equation (2.3).

To calculate either standard deviation in Excel, you can first find the variance with the VAR.S or VAR.P function and then take its square root. Alternatively, you can find it directly with the STDEV.S (sample) or STDEV.P (population) function.

We used these functions, actually the sample versions, in cells F4 and F5 of Figure 2.10 for the baseball salaries. The huge value for variance is in dollars squared, whereas the standard deviation is in dollars, about $6.233 million. By the way, you might argue that the population versions of these parameters should be used instead, and you would have a point. After all, the data set includes all 2018 baseball players. However, you can check that with 877 players, the difference between the sample and population measures is negligible.

Denominators of Variance Formulas

It is traditional to use the capital letter N for the population size and lowercase n for the sample size, but this distinction is not important in this chapter. Furthermore, there is a technical reason why the sample variance uses n 2 1 in the denominator, not n, and this is explained in a later chapter. In any case, the difference is negligible when n is large. Excel implements both of these formulas. You can use the VAR.S function to obtain the sample variance (denominator n 2 1), and you can use the VAR.P function to obtain the population variance (denominator n).

Technical Note

New VAR and STDEV Functions

Microsoft added four new functions, VAR.S, VAR.P, STDEV.S, and STDEV.P, in Excel 2010. These are identical to the “old” VAR, VARP, STDEV, and STDEVP functions, and you can use either the old or the new versions.

Excel Tip

The data in Figure 2.11 help clarify these concepts. It is in the file Variability.xlsx. (It will help if you open this file and look at its formulas as you read this.) The variable Diameter1 on the left has relatively low variability; its 10 values vary closely around its mean of approximately 100, found in cell A16 with the AVERAGE function. To show how variance is calculated, we explicitly calculated the 10 squared deviations from the mean in column B. Then either variance, sample or population, can be calculated (in cells A19 and A22) as the sum of squared deviations divided by 9 or 10. Alternatively, they can be calculated more directly (in cells B19 and B22) with Excel’s VAR.S and VAR.P functions. Next, either standard deviation, sample or pop- ulation, can be calculated as the square root of the corresponding variance or with Excel’s STDEV.S or STDEV.P functions.

The calculations are exactly the same for Diameter2 on the right. This variable also has mean approximately equal to 100, but its observations vary much more around 100 than the observations for Diameter1. As expected, this increased variability is obvious in a comparison of the variances and standard deviations for the two suppliers.

This example also indicates why variability is important. Imagine that you are about to buy 10 parts from one of two suppliers, and you want each part’s diameter to be close to 100 cen- timeters. Furthermore, suppose that Diameter1 in the example represents 10 randomly selected parts from supplier 1, whereas Diameter2 represents 10 randomly selected parts from Supplier 2. You can see that both suppliers are very close to the target of 100 on average, but the increased

5 4 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Variability is usually the enemy. Being close to a target value on average is not good enough if there is a lot of variability around the target.

09953_ch02_ptg01_037-083.indd 54 04/03/19 10:41 PM

variability for Supplier 2 makes this supplier much less attractive. There is a famous saying in operations management: Variability is the enemy. This example illustrates exactly what this saying means.

Empirical Rules for Interpreting Standard Deviation Now you know how to calculate the standard deviation, but there is a more important question: How do you interpret its value? Fortunately, the standard deviation often has a very natural interpretation, which is why it is quoted so frequently. This interpretation can be stated as three empirical rules. (“Empirical” means that they are based on commonly observed data, as opposed to theoretical mathematical arguments.) Specifically, if the values of this variable are approximately normally distrib- uted (symmetric and bell-shaped), then the following rules hold:

• Approximately 68% of the observations are within one standard deviation of the mean, that is, within the interval X { s. • Approximately 95% of the observations are within two standard deviations of the mean, that

is, within the interval X { 2s. • Approximately 99.7% of the observations—almost all of them—are within three standard

deviations of the mean, that is, within the interval X { 3s. Fortunately, many variables in real-world data are approximately normally distributed,

so these empirical rules correctly apply. (The normal distribution is discussed in depth in Chapter 5.)

Figure 2.11 Calculating Variance and Standard Deviation

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

A B C D E F Low variability supplier High variability supplier

Diameter 1 Sq dev from qS2retemaiDnaem dev from mean 694438.912.301140016.616.201 693931.1466.39125013.0152.301 616374.23478.021106286.3143.69 695457.30162.011163502.4172.69 696970.79213.711163029.3177.301 633441.30132.011129207.654.79 651752.27845.07167803.322.89 639575.566335.93148304.767.201 613756.890122.331144313.265.101 698073.319.101146035.361.89

MeannaeM 100.074930.001

Sample e variancelpmaSecnairav 3563.6373563.6378901.98901.9

noitalupoPecnaira Population v variance 7827.2667827.2668891.88891.8

Sample standard elpmaSnoitaived standard deviation 1631.721631.722810.32810.3

Population standard noitalupoPnoitaived standard deviation 5347.525347.524368.24368.2

These empirical rules give a concrete meaning to standard deviation for symmetric, bell-shaped distributions. However, they tend to be less accurate for skewed distributions.

2-4 Summarizing Numeric Variables 5 5

09953_ch02_ptg01_037-083.indd 55 04/03/19 10:41 PM

As an example, if the parts supplied by the suppliers in Figure 2.11 have diameters that are approximately normally dis- tributed, then the intervals in the empirical rules for supplier 1 are about 100 { 3, 100 { 6, and 100 { 9. Therefore, about 68% of this supplier’s parts will have diameters from 97 to 103, 95% will have diameters from 94 to 106, and almost none will have diameters below 91 or above 109. Obviously, the situation for supplier 2 is much worse. With a standard deviation slightly larger than 25, the second empirical rule implies that about 1 out of every 20 of this supplier’s parts will be below 50 or above 150. It is clear that supplier 2 has to reduce its variability. In fact, this is exactly what almost all suppliers are contin- uously trying to do: reduce variability.

Returning to the baseball data, Figure 2.10 indicates that the standard deviation of salaries is about $6.233 million. (The variance is shown, but because it is in squared dollars, it is a huge value with no meaningful interpretation.) Can the empirical rules be applied to these baseball salaries? The answer is that you can always try, but if the salaries are not at least approximately normally distributed, the rules won’t be very accurate. And because of obvious skewness in the salary data, due to the stars with huge salaries, the assumption of a normal distribution is not a good one.

Nevertheless, the rules are checked in Figure 2.12. For each of the three rules, the lower and upper endpoints of the corre- sponding interval are found in columns I and J. Right away there are problems. Because the standard deviation is larger than the mean, all three lower endpoints are negative, which automatically means that there can be no salaries below them. But con- tinuing anyway, the COUNTIF was used, again with concatenation, to find the number of salaries above the upper endpoints in column L, and the corresponding percentages appear in column N. Finally, subtracting columns M and N from 100% in column O gives the percentages between the endpoints. These three percentages, according to the empirical rules, should be about 68%, 95%, and 99.7%. Rules 2 and 3 are not way off, but rule 1 isn’t even close.

Usefulness of Standard Deviation

Variability is an important property of any numeric variable, and there are several measures for quantifying the amount of variability. Of these, standard deviation is by far the most frequently quoted measure. It is measured in the same units as the variable, it has a long tradition, and, at least for many data sets, it obeys the empirical rules discussed here. These empirical rules give a very specific meaning to standard deviation.

Fundamental Insight

Figure 2.12 Empirical Rules for Baseball Salaries

1 2 3 4 5

H I J K L M N O Do empirical rules apply?

Lower endpoint Upper endpoint # below lower # above upper % below lower % above upper % between –$1,719,868 –$7,952,504

–$14,185,141

$10,745,405 $16,978,041 $23,210,677

0 0% 14% 7% 2%

86.20% 93.04% 97.95%

0 0% 0

121 61 18 0%

Rule 1 Rule 2 Rule 3

The point of these calculations is that even though the empirical rules give substantive meaning to the standard deviation for many variables, they should be applied with caution, especially when the data are clearly skewed.

Before leaving variance and standard deviation, you might ask why the deviations from the mean are squared in the definition of variance. Why not simply take absolute deviations from the mean? For example, if the mean is 100 and two observations have values 95 and 105, then each has a squared deviation of 25, but each has an absolute deviation of only 5. Wouldn’t this latter value be a more natural mea- sure of variability? Intuitively, it would, but there is a long history of using squared deviations. They have many attractive theoret- ical properties that are not shared by absolute deviations. Still, some analysts quote the mean absolute deviation (abbreviated as MAD) as another measure of variability, particularly in time series analysis. It is defined as the average of the absolute deviations.

5 6 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Don't be surprised if the empirical rules don't hold for skewed data. The rules are based on the normal distribution, which is symmetric and bell-shaped.

Formula for Mean Absolute Deviation

MAD 5 an i 5 1

kXi 2 mean k

n (2.4)

09953_ch02_ptg01_037-083.indd 56 04/03/19 10:41 PM

There is another empirical rule for MAD: For many (but not all) variables, the standard deviation is approximately 25% larger than MAD, that is, s ^ 1.25*MAD. Fortunately, Excel has a function, AVEDEV, that performs the calculation in Equa- tion (2.4). Using it for the baseball salaries in Figure 2.10, you can see that MAD is about $4.553 million. If this is multiplied by 1.25, the result is slightly over $5.69 million, which is not too far from the standard deviation.

Measures of Shape There are two final measures of a distribution you will hear occasionally: skewness and kurtosis. Each of these has not only an intuitive meaning but also a specific numeric measure. We have already mentioned skewness in terms of the baseball sala- ries. It occurs when there is a lack of symmetry. A few stars have really large salaries, and no players have really small salaries. More specifically, the largest salaries are much farther to the right of the mean than the smallest salaries are to the left of the mean. This lack of symmetry is usually apparent from a histogram of the salaries, discussed in the next section. We say that these salaries are skewed to the right (or positively skewed) because the skewness is due to the really large salaries. If the skewness were due to really small values, as might occur with temperature lows in Antarctica, then we would call it skewness to the left (or negatively skewed).

In either case, a measure of skewness can be calculated with Excel’s SKEW function. For the baseball data, it is approx- imately 2.1, as shown in Figure 2.10. You don’t need to know exactly what this value means. Simply remember that (1) it is positive when there is skewness to the right, (2) it is negative when there is skewness to the left, (3) it is approximately zero when the data are symmetric, and (4) its magnitude increases as the degree of skewness increases.

The other measure, kurtosis, has to do with the “fatness” of the tails of the distribution relative to the tails of a normal dis- tribution. Remember from the third empirical rule that a normal distribution has almost all of its observations within three standard deviations of the mean. In contrast, a distri- bution with high kurtosis has many more extreme observations. In reality, this is much more important than you might imagine. For example, many researchers believe the Wall Street meltdown in late 2008 was at least partly due to financial analysts relying on the normal distribution, whereas in reality the actual distribution had much fatter tails. More specifically, financial analysts followed complex mathematical models that indicated really extreme events would virtually never occur. Unfortunately, a number of extreme events did occur, and they sent the economy into a deep recession.1

Although kurtosis can be calculated in Excel with the KURT function (it is about 4.1 for the baseball salaries), we won’t have any use for this measure in the book. Nevertheless, when you hear the word kurtosis, think of fat tails and extreme events. And if you plan to work on Wall Street, you should definitely learn more about kurtosis.

Numeric Summary Measures in the Status Bar You might have noticed that summary measures sometimes appear automatically in the status bar at the bottom of your Excel window. The rule is that if you select multiple cells (in a single column or even in multiple columns), selected summary measures appear for the selected cells. (Nothing appears if only a single cell is selected.) These can be very handy for quick lookups. Also, you can choose the summary measures that appear by right-clicking the status bar and selecting your favorites.

Numeric Summary Measures with Add-Ins All the summary measure just discussed can be calculated with built-in Excel functions such as AVERAGE and STDEV.S. However, Excel add-ins can be used to speed up the process. Specifically, Palisade’s StatTools add-in, part of their Decision Tools Suite, is discussed in an appendix to this chapter. StatTools can generate summary measures for one or more numeric variables in seconds, and it is also capable of more advanced statistical analysis, such as regression and time series analysis, topics of later chapters.

2-4 Summarizing Numeric Variables 5 7

1 The popular book The Black Swan, by Nassim Nicholas Taleb, is all about extreme events and the trouble they can cause.

2-4b Charts for Numeric Variables There are many graphical ways to indicate the distribution of a numeric variable, but the two we prefer and discuss in this subsection are histograms and box plots (also called box-whisker plots). Each of these is useful primarily for cross-sectional variables. If they are used for time series variables, the time dimension is lost. Therefore, we discuss time series graphs for time series variables separately in the next section.

Kurtosis is all about extreme events—the kind that occurred in late 2008 and sent Wall Street into a panic.

09953_ch02_ptg01_037-083.indd 57 04/03/19 10:41 PM

5 8 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Histograms A histogram is the most common type of chart for showing the distribution of a numeric variable. It is based on binning the variable—that is, dividing it up into discrete catego- ries. The histogram is then a column chart of the counts in the various categories, usu- ally with no gaps between the vertical bars. In general, a histogram is useful for showing the shape of a distribution. Of particular interest is whether the distribution is symmetric or skewed.

histograms Versus Summary Measures

It is important to remember that each of the summary measures we have dis- cussed for a numeric variable—mean, median, standard deviation, and others— describes only one aspect of a numeric variable. In contrast, a histogram provides the complete picture. It indicates the “center” of the distribution, the variability, the skewness, and other aspects, all in one chart.

Fundamental Insight

The steps for creating a histogram are straightforward:

1. Decide how many bins to use and what the endpoints of the bins should be. For exam- ple, with exam scores between 51 and 100, you might choose 10 bins: 51 to 55, 56 to 60, 61 to 65, and so on.

2. Count the number of observations in each bin. There are Excel functions for doing this, including COUNTIFS and FREQUENCY. (We prefer COUNTIFS. The FREQUENCY function is an array function and is tricky to use.)

3. Create a column chart of these counts and remove the gaps between the bars.

The steps are fairly easy, but the whole process is tedious, and it is easy to make mis- takes. Therefore, numerous tools have been developed for creating histograms quickly and easily. For Excel users, these include the following:

1. Microsoft added histograms to its list of chart types in Excel 2016. (It was recently added to the latest version of Excel for the Mac as well.) You can find this in the Statis- tics Chart group (the icon looks like a mini-histogram) on the Insert ribbon. Just select any cell in the data column, click the Histogram button in this group, and you get an automatic histogram.

2. The Analysis ToolPak add-in includes a histogram tool. Just be warned that its results are not very pretty.

Each of these requires a choice of bins: how many and which endpoints. Fortunately, researchers have devised good rules of thumb for bin selection, so that you don't have to worry about the details. For example, Excel's built-in histogram chooses the bins automat- ically, but then you can right-click the horizontal axis and choose Format Axis to modify the bins, at least to some extent. Fortunately, the bin selection is usually not crucial. The real purpose of a histogram is show the shape of the distribution of data values, and the same basic shape emerges with any reasonable set of bins.

Figure 2.13 shows a histogram of the 2018 baseball salaries. It was created with Excel's fairly new histogram chart type. (To create it, select any cell in the Salary column and then click the histogram chart type on the Insert ribbon.) The bins are selected auto- matically, but this makes little difference for the salary data. You can see exactly what you need to see: The baseball salaries are heavily skewed to the right. Nevertheless, if you right-click the horizontal axis and choose Format Axis, the pane in Figure 2.14 opens. This allows you to change the bins, at least to some extent.

09953_ch02_ptg01_037-083.indd 58 04/03/19 10:41 PM

2-4 Summarizing Numeric Variables 5 9

Figure 2.13 Histogram of 2018 Baseball Salaries

Figure 2.14 Options for Changing Bins

Histogram of 2018 Salaries 600

500

400

300

200

100

($ 54

5, 00

0, ...

($ 2,

84 5,

00 0,

...

($ 5,

14 5,

00 0,

...

($ 7,

44 5,

00 0,

...

($ 9,

74 5,

00 0,

...

($ 12

,0 45

,0 00

,.. .

($ 14

,3 45

,0 00

,.. .

($ 16

,6 45

,0 00

,.. .

($ 18

,9 45

,0 00

,.. .

($ 21

,2 45

,0 00

,.. .

($ 23

,5 45

,0 00

,.. .

($ 25

,8 45

,0 00

,.. .

($ 28

,1 45

,0 00

,.. .

($ 30

,4 45

,0 00

,.. .

($ 32

,7 45

,0 00

,.. .

Box Plots A box plot (also called a box-whisker plot) is an alternative type of chart for showing the distribution of a variable. For the distribution of a single variable, a box plot is not nearly as popular as a histogram, but as you will see in the next chapter, side-by-side box plots are very useful for comparing distributions, such as salaries for men versus salaries for women. As with histograms, box plots are “big picture” charts. They show you at a glance some of the key features of a distribution.

Like histograms, box plots were added to Excel in Excel 2016, and they are now avail- able in Excel for Windows and Excel for Mac. They are in the same chart group as histo- grams on the Insert ribbon. The box plot of 2018 baseball salaries appears in Figure 2.15. To create it, select the entire Salary column before clicking the box plot option on the Insert ribbon. (If you select only a single cell, you will get box plots broken down by Position,

09953_ch02_ptg01_037-083.indd 59 04/03/19 10:41 PM

6 0 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

the type illustrated in the next chapter.) You can also delete the horizontal axis label. The label shown has no significance.

Box Plot of 2018 Salaries

$35,000,000

$40,000,000

$30,000,000

$25,000,000

$20,000,000

$15,000,000

$10,000,000

$5,000,000

Figure 2.15 Box Plot of Baseball Salaries

A box plot has the following elements:

1. The box itself, from bottom to top, extends from the first quartile to the third quartile, so it contains the middle 50% of the data. (Box plots from some software packages are shown horizontally rather than vertically. Then the width of the box indicates the middle 50% of the data.)

2. The horizontal line inside the box represents the median, and the x inside the box represents the mean.

3. The lines from either end of the box, also called whiskers, extend as far as the most distant data value within 1.5 IQRs (interquartile ranges) of the box.

4. Any data values beyond the whiskers are called outliers and are shown as individual points.

The box plot of salaries in Figure 2.15 should now make more sense. It is typical of an extremely right-skewed distribution. The mean is much larger than the median, there is virtually no whisker out of the low side of the box (because the first quartile is barely above the minimum value—remember all the players earning $545,000?), and there are many outliers on the high side (the highest paid players). In fact, many of these outliers overlap one another. You can decide whether you prefer the histogram of salaries to the box plot or vice versa, but both are clearly telling the same story.

Box plots have existed for several decades, and they are probably more popular now than ever. The implementation of box plots in Excel is just one version of what you might see. Some software packages vary the width of the box to indicate some other feature of the distribution. (The width of the box is irrelevant in Excel’s box plots.) Nevertheless, they all follow the same basic rules and provide the same basic information.

Box plots Versus histograms

Box plots and histograms are complementary ways of displaying the distribution of a numeric variable. Although histograms are much more popular and are argu- ably more intuitive, box plots are still very informative. Besides, side-by-side box plots are very useful for comparing two or more populations, as illustrated in the next chapter.

Fundamental Insight

09953_ch02_ptg01_037-083.indd 60 04/03/19 10:41 PM

2-4 Summarizing Numeric Variables 6 1

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 6. The file P02_06.xlsx lists the average time (in minutes)

it takes citizens of 379 metropolitan areas to travel to work and back home each day. a. Create a histogram of the daily commute times. b. Find the most representative average daily commute

time across this distribution. c. Find a useful measure of the variability of these aver-

age commute times around the mean. d. The empirical rule for standard deviations indi-

cates that approximately 95% of these average travel times will fall between which two values? For this particular data set, is this empirical rule at least approximately correct?

7. The file P02_07.xlsx includes data on 204 employees at the (fictional) company Beta Technologies. a. Indicate the data type for each of the six variables

included in this data set. b. Create a histogram of the Age variable. How would

you characterize the age distribution for these employees?

c. What proportion of these full-time Beta employees are female?

d. Find appropriate summary measures for each of the numeric variables in this data set.

e. For the Salary variable, explain why the empiri- cal rules for standard deviations do or do not apply.

8. The file P02_08.xlsx contains data on 500 shipments of one of the computer components that a company man- ufactures. Specifically, the proportion of items that are defective is listed for each shipment. a. Create a histogram that will help a production man-

ager understand the variation of the proportion of defective components in the company’s shipments.

b. Is the mean or median the most appropriate measure of central location for this data set? Why?

c. Do the empirical rules for standard deviations apply? Can you tell, or at least make an educated guess, by looking at the shape of the histogram? Why?

9. The file P02_09.xlsx lists the times required to service 200 consecutive customers at a (fictional) fast-food restaurant. a. Create a histogram of the customer service times. How

would you characterize the distribution of service times? b. Calculate the mean, median, and first and third quar-

tiles of this distribution. c. Which measure of central tendency, the mean or the

median, is more appropriate in describing this distri- bution? Explain your reasoning.

d. Find and interpret the variance and standard deviation of these service times.

e. Are the empirical rules for standard deviations appli- cable for these service times? If not, explain why. Can you tell whether they apply, or at least make an educated guess, by looking at the shape of the histo- gram? Why?

10. The file P02_10.xlsx contains midterm and final exam scores for 96 students in a corporate finance course. a. Create a histogram for each of the two sets of exam

scores. b. What are the mean and median scores on each of these

exams? c. Explain why the mean and median values are different

for these data. d. Based on your previous answers, how would you

characterize this group’s performance on the midterm and on the final exam?

e. Create a new column of differences (final exam score minus midterm score). A positive value means the stu- dent improved, and a negative value means the student did the opposite. What are the mean and median of the differences? What does a histogram of the differ- ences indicate?

11. The file P02_11.xlsx contains data on 148 houses that were recently sold in a (fictional) suburban community. The data set includes the selling price of each house, along with its appraised value, square footage, number of bedrooms, and number of bathrooms. a. Which of these variables are continuous? Which are

discrete? b. Create histograms for the appraised values and selling

prices of the houses. How are these two distributions similar? How are they different?

c. Find the maximum and minimum sizes (measured in square footage) of all sample houses.

d. Find the house(s) at the 80th percentile of all sam- ple houses with respect to appraised value. Find the house(s) at the 80th percentile of all sample houses with respect to selling price.

e. What are the typical number of bedrooms and the typ- ical number of bathrooms in this set of houses? How do you interpret the word “typical”?

12. The file P02_12.xlsx includes data on the 50 top grad- uate programs in the United States, according to a 2009 U.S. News & World Report survey. a. Indicate the type of data for each of the 10 variables

considered in the formulation of the overall ranking. b. Create a histogram for each of the numeric variables

in this data set. Indicate whether each of these dis- tributions is approximately symmetric or skewed. Which, if any, of these distributions are skewed to the right? Which, if any, are skewed to the left?

c. Identify the schools with the largest and smallest annual out-of-state tuition and fee levels.

d. Find the annual out-of-state tuition and fee levels at each of the 25th, 50th, and 75th percentiles for these schools.

09953_ch02_ptg01_037-083.indd 61 04/03/19 10:41 PM

6 2 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

e. (Requires Excel version 2016 or later) Create a box plot to characterize this distribution of these MBA salaries. Is this distribution essentially symmetric or skewed? If there are any outliers on either end, which schools do they correspond to?

13. The file P02_13.xlsx contains the thickness (in centime- ters) of 252 mica pieces. A piece meets specifications if its thickness is between 7 and 15 centimeters. a. What fraction of mica pieces meets specifications? b. Are the empirical rules for standard deviations at least

approximately valid for these data? Can you tell, or at least make an educated guess, by looking at a histo- gram of the data?

c. If the histogram of the data is approximately bell- shaped and you want about 95% of the observations to meet specifications, is it sufficient for the average and standard deviation to be, at least approximately, 11 and 2 centimeters, respectively?

14. (Requires Excel version 2016 or later) Recall that the file Supermarket Transactions.xlsx contains over 14,000 transactions made by super-market customers over a period of approximately two years. Using these data, create side-by-side box plots for revenues broken down by state or province. Are these distributions essen- tially symmetric or skewed? Note that these box plots include revenues from countries besides the United States. Do whatever it takes to create side-by-side box plots of revenue for only states within the United States.

15. (Requires Excel version 2016 or later) Recall that the file Baseball Salaries.xlsx contains data on 877 MLB players in the 2018 season. Using these data, create side-by-side box plots to characterize the distribution of salaries of all pitchers versus all non-pitchers. Summa- rize your findings.

16. The file P02_16.xlsx contains traffic data from 256 weekdays on four variables. Each variable lists the num- ber of vehicle arrivals to a tollbooth during a specific five-minute period of the day. a. Create a histogram of each variable. How would you

characterize and compare these distributions? b. Find a table of summary measures for these variables

that includes (at least) the means, medians, standard

deviations, first and third quartiles, and 5th and 95th percentiles. Use these to compare the arrival process at the different times of day.

Level B 17. The file P02_17.xlsx contains salaries of 200 recent

graduates from a (fictional) MBA program. a. What salary level is most indicative of those earned

by students graduating from this MBA program this year?

b. Do the empirical rules for standard deviations apply to these data? Can you tell, or at least make an edu- cated guess, by looking at the shape of the histogram? Why?

c. If the empirical rules apply here, between which two numbers can you be about 68% sure that the salary of any one of these 200 students will fall?

d. If the MBA program wants to make a statement such as “Some of our recent graduates started out making X dollars or more, and almost all of them started out making at least Y dollars” for their promotional mate- rials, what values of X and Y would you suggest they use? Defend your choice.

e. As an admissions officer of this MBA program, how would you proceed to use these findings to market the program to prospective students?

18. The file P02_18.xlsx contains daily values of the Standard & Poor’s 500 Index for a five-year period. It also contains percentage changes in the index from each day to the next. a. Create a histogram of the percentage changes and

describe its shape. b. Check the percentage of these percentage changes that

are more than k standard deviations from the mean for k 5 1, 2, 3, 4, and 5. Are these approximately what the empirical rules indicate or are there “fat” tails? Do you think this has any real implications for the finan- cial markets? (Note that we have discussed the empir- ical rules only for k 5 1, 2, and 3. For k 5 4 and 5, they indicate that only 0.006% and 0.0001% of the observations should be this distant from the mean.)

2-5 Time Series Data When we analyze time series variables, summary measures such as means and standard deviations and charts such as histograms and box plots are often inappropriate. Our main interest in time series variables is how they change over time, and this information is lost in traditional summary measures, histograms, and box plots. Imagine, for example, that you are interested in daily closing prices of a stock that has historically been between 20 and 60. If you create a histogram with a bin such as 45 to 50, you will get a count of all daily closing prices in this interval—but you won’t know when they occurred. The histo- gram is missing a key feature: time. Similarly, if you report the mean of a time series such

09953_ch02_ptg01_037-083.indd 62 04/03/19 10:41 PM

2-5 time Series Data 6 3

as the monthly Dow Jones average over the past 40 years, you will get a measure that is not relevant for the current and future values of the Dow.

Therefore, we turn to a different but very intuitive type of graph called a time series graph. This is a graph of the values of one or more time series, using time on the horizon- tal axis, and it is always the place to start a time series analysis. We illustrate some possi- bilities in the following example.

EXAMPLE

2.4 CRIME IN THE UNITED STATES The file Crime in US.xlsx contains annual data on violent and property crimes for the years 1960 to 2010. Part of the data is listed in Figure 2.16. This shows the number of crimes. The rates per 100,000 population are not shown, but they can be calcu- lated easily. Are there any apparent trends in this data? If so, are the trends the same for the different types of crime?

Figure 2.16 Crime Data

1 2 3 4 5 6 7 8 9

A B C D E F G H I J K

Ye ar

Vi ol

en t c

rim e

to ta

Po pu

la tio

M ur

de r a

nd no

nn eg

lig en

t m

an sl

au gh

te r

Fo rc

ib le

ra pe

Ro bb

er y

Ag gr

av at

ed as

sa ul

Pr op

er ty

c rim

e to

ta l

Bu rg

la ry

La rc

en y-

th ef

M ot

or v

eh ic

le th

ef t

179,323,175 288,460 1,855,400 182,992,000 289,390 1,913,000 185,771,000 301,510 2,089,600 188,483,000 316,970 2,297,800 191,141,000 364,220 2,514,400 193,526,000 387,390 2,572,600 195,576,000 430,180 2,822,000 197,457,000 499,930 3,111,600

1960 1961 1962 1963 1964 1965 1966 1967 1968 199,399,000 595,010

9,110 8,740 8,530 8,640 9,360 9,960

11,040 12,240 13,800

17,190 17,220 17,550 17,650 21,420 23,410 25,820 27,620 31,670

107,840 106,670 110,860 116,470 130,390 138,690 157,990 202,910 262,840

154,320 156,760 164,570 174,210 203,050 215,330 235,330 257,160 286,700

3,095,700 3,198,600 3,450,700 3,792,500 4,200,400 4,352,000 4,793,300 5,403,500 6,125,200

912,100 949,600 994,300

1,086,400 1,213,200 1,282,500 1,410,100 1,632,100 1,858,900 3,482,700

328,200 336,000 366,800 408,300 472,800 496,900 561,200 659,800 783,600

Objective To see how time series graphs help to detect trends in crime data.

Solution It is easy to create a time series graph in Excel. Here are the steps:

1. Select one or more columns with time series data. 2. Optionally, select a column with dates if there is one. These dates are used to label the horizontal axis. 3. Select one of the line chart types, such as the line chart with markers, from the Insert ribbon.

This procedure will usually get you nearly what you want, and then you can modify the chart to suit your needs, such as adding a title.

Formatting Long Variable Names

Note the format of the variable names in row 1. If you have long variable names, one possibility is to align them vertically and check the Wrap Text option. (These are both available through the Format Cells command, which can be accessed by right-clicking any cell or pressing Ctrl+1.) With these changes, the row 1 labels are neither too tall nor too wide.

Excel Tip

09953_ch02_ptg01_037-083.indd 63 04/03/19 10:41 PM

These steps were used to create the time series graphs in Figures 2.17, 2.18, and 2.19. The graph in Figure 2.17 shows that violent and property crimes both increased sharply until the early 1990s and have gradually decreased since then. However, the time series population series in Figure 2.18 indicates that the U.S. population has increased steadily since 1960, so it is possible that the trend in crime rates is different than the trends in Figure 2.17. This is indeed true, as seen in Figure 2.19. It shows the good news that the crime rate has been falling since its peak in the early 1990s.2

Date variable

When you proceed as above, the date variable is sometimes treated as another time series variable, with a line of its own. The happens, for example, in the crime data set with the Year variable. It looks numeric, so Excel treats it as another time series variable. In this case, select the chart, click Select Data on the Chart Tools Design ribbon, delete the date series in the left pane, and edit the right pane, selecting the date variable range for the horizontal axis.

Excel Tip

Secondary axis

If you select two time series variables with very different magnitudes, the smaller will get swamped by the larger in the time series graph. In this case, right-click the smaller series, select Format Data Series, and check Second- ary Axis. This allows you to see the smaller series on a separate scale.

Excel Tip

The whole purpose of time series graphs is to detect historical patterns in the data. In this crime example, you are looking for broad trends.

2500000

2000000

1500000

1000000

Vi ol

en t c

ri m

e to

ta l/

Cr im

e D

at a

500000

19 60

19 62

19 64

19 66

19 68

19 70

19 72

19 74

19 76

19 78

19 80

19 82

19 84

19 86

19 88

19 90

19 92

19 94

19 96

19 98

20 00

20 02

20 04

20 06

20 08

20 10

14000000

Time Series

12000000

10000000

8000000

6000000

4000000

2000000

Pr op

er ty

c ri

m e

to ta

l/ Cr

im e

D at

Violent crime total/Crime Data Property crime total/Crime Data

Figure 2.17 Total Violent and Property Crimes

2 Why did this occur? One compelling reason was suggested by Levitt and Dubner in their popular book Freakonomics. You can read their somewhat controversial analysis to see if you agree.

6 4 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

09953_ch02_ptg01_037-083.indd 64 04/03/19 10:41 PM

We also created two more time series graphs that appear in Figures 2.20 and 2.21. The first shows the crime rates for the various types of violent crimes, and the second does the same for property crimes. The patterns (up, then down) are similar for each type of crime, but they are certainly not identical. For example, the larceny-theft and motor vehicle theft rates both peaked in the early 1990s, but the burglary rate was well in decline by this time. Finally, Figure 2.20 indicates one problem with having multiple time series variables on a single graph—any variable with small values can be dominated by variables with much larger values. It might be a good idea to create two separate graphs for these four variables, with murder and rape on one and robbery and aggravated assault on the other. Then you could see the murder and rape patterns more clearly, and each of the graphs would be less cluttered.

50000000

100000000

150000000

200000000

250000000

300000000

350000000

19 60

19 62

19 64

19 66

19 68

19 70

19 72

19 74

19 76

19 78

19 80

19 82

19 84

19 86

19 88

19 90

19 92

19 94

19 96

19 98

20 00

20 02

20 04

20 06

20 08

20 10

Time Series of Population/Crime Data Figure 2.18 Population Totals

Figure 2.19 Violent and Property Crime Rates

19 60

19 62

19 64

19 66

19 68

19 70

19 72

19 74

19 76

19 78

19 80

19 82

19 84

19 86

19 88

19 90

19 92

19 94

19 96

19 98

20 00

20 02

20 04

20 06

20 08

20 10

800

700

600

500

Vi ol

en t c

ri m

e ra

te /C

ri m

e D

at a

400

300

200

100

6000

Time Series

5000

4000

3000

Pr op

er ty

c ri

m e

ra te

/C ri

m e

D at

2000

1000

Violent crime rate/Crime Data Property crime rate/Crime Data

2-5 time Series Data 6 5

09953_ch02_ptg01_037-083.indd 65 04/03/19 10:41 PM

Sparklines

One feature introduced in Excel 2010 is the sparkline. This is a mini-chart embedded in a cell. Although it applies to any kind of data, it is especially useful for time series data. Try the following. Open a file, such as the problem file P03_30.xlsx, that has multiple time series—one per column. Highlight the cell below the last time series value of the first time series and click the Line button in the Sparklines group on the Insert ribbon. In the resulting dialog box, select the data in the first time series. You will get a mini-time-series graph in the cell. Now copy this cell across for the other time series, and increase the row height to expand the graphs. Change any of the time series values to see how the sparklines change automatically. We suspect that these instant sparkline graphs are becoming quite popular.

Excel Tool

1200000

1000000

800000

600000

400000

200000

19 60

19 62

19 64

19 66

19 68

19 70

19 72

19 74

19 76

19 78

19 80

19 82

19 84

19 86

19 88

19 90

19 92

19 94

19 96

19 98

20 00

20 02

20 04

20 06

20 08

20 10

Time Series

Murder and nonnegligent manslaughter/Crime Data Forcible rape/Crime Data

Robbery/Crime Data

Aggravated assault/Crime Data

Figure 2.20 Rates of Violent Crime Types

Figure 2.21 Rates of Property Crime Types

3500

3000

2500

2000

1500

1000

500

19 60

19 62

19 64

19 66

19 68

19 70

19 72

19 74

19 76

19 78

19 80

19 82

19 84

19 86

19 88

19 90

19 92

19 94

19 96

19 98

20 00

20 02

20 04

20 06

20 08

20 10

Burglary rate/Crime Data Larceny-theft rate/Crime Data Motor vehicle theft rate/ Crime Data

Time Series

As mentioned earlier, traditional summary measures such as means, medians, and standard deviations are often not very meaningful for time series data, at least not for the original data. However, it is often useful to find differences or percentage changes in the data from period to period and then report traditional summary measures of these. The following example illustrates these ideas.

6 6 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

09953_ch02_ptg01_037-083.indd 66 04/03/19 10:41 PM

2-5 time Series Data 6 7

EXAMPLE

2.5 THE DOW JONES INDEX The Dow Jones Industrial Average (DJIA or simply “the Dow”) is an index of 30 large publicly traded U.S. stocks and is one of the most quoted stock indexes. The file DJIA Monthly Close.xlsx contains monthly values of the Dow from 1950 through early 2018. What is a useful way to summarize the data in this file?

Objective To find useful ways to summarize the monthly Dow data.

Solution A time series graph and a few summary measures of the Dow appear in Figure 2.22. The graph clearly shows a gradual increase through the early 1990s (except for Black Monday in 1987), a sharp increase through the rest of the 1990s, some huge swings through 2008, and a sharp increase since then. The mean (4781), the median (1210), and any of the other traditional summary measures are of historical interest at best.

Figure 2.22 Summary Measures and Graph of the Dow

Mean Std Dev Median Quartile 1 Quartile 3

4781.3 5687.3 1209.5

806.6 9311.2

1 2 3 4 5 6 7 8 9

D E F G H I J K L M

10 11 12 13 14 15

Summary measures

Ja n-

50 Se

p- 53

M ay

-5 7

Ja n-

61 Se

p- 64

Se p-

M ay

-6 8

M ay

-7 9

M ay

-9 0

M ay

-0 1

M ay

-1 2

Ja n-

30,000.00

25,000.00

20,000.00

15,000.00

10,000.00

5,000.00

0.00

Closing Value

In situations like this, it is useful to look at percentage changes in the Dow. These have been calculated in the file and have been used to create the summary measures and time series graph in Figure 2.23. The graph shows that these percentage changes have fluctuated around zero, sometimes with wild swings (like Black Monday). Actually, the mean and median of the percentage changes are slightly positive, about 0.67% and 0.85%, respectively. In addition, the quartiles show that 25% of the changes have been less than 21.61% and 25% have been greater than 3.22%. Finally, the empirical rules indicate, for example, that about 95% of the percentage changes over this period should be no more than two standard deviations (8.18%) from the mean. You can check that the actual percentage within two standard deviations of the mean is close to 95%, so this empirical rule applies very well.3

3 One of the problems asks you to check whether all three of the empirical rules apply to similar stock price data. The extreme tails are where there are some surprises.

09953_ch02_ptg01_037-083.indd 67 04/03/19 10:41 PM

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 19. The file P02_19.xlsx lists annual percentage changes in

the Consumer Price Index (CPI) from 1914 through 2017. Find and interpret the first and third quartiles and the interquartile range for these annual percentage changes. Discuss whether these are meaningful summary measures for this time series data set. Suppose that the data set listed the actual CPI values, not percentage changes, for each year. Would the quartiles and interquartile range be mean- ingful in this case? Why or why not?

20. The Consumer Confidence Index (CCI) attempts to measure people’s feelings about general business condi- tions, employment opportunities, and their own income prospects. Monthly average values of the CCI are listed in the file P02_20.xlsx. a. Create a time series graph of the CCI values. b. Have U.S. consumers become more or less confident

through time? c. How would you explain recent variations in the over-

all trend of the CCI? 21. The file P02_21.xlsx contains monthly interest rates on

30-year and 15-year fixed-rate mortgages in the United States from 1991 to 2017. What conclusion(s) can you draw from a time series graph of these mortgage rates? Specifically, what has been happening to mortgage rates in general, and how does the behavior of the 30-year rates compare to the behavior of the 15-year rates?

22. The file P02_22.xlsx contains annual trade balances (exports minus imports) from 1980 to 2017. a. Create a times series graph for each of the three time

series in this file.

b. Characterize recent trends in the U.S. balance of trade figures using your time series graphs.

23. What has happened to the total number and average size of farms in the U.S. since the middle of the 20th century? Answer this question by creating a time series graph of the data from the U.S. Department of Agriculture in the file P02_23.xlsx. Is the observed result consistent with your knowledge of the structural changes within the U.S. farming economy?

24. Is educational attainment in the United States on the rise? Explore this question by creating time series graphs for each of the variables in the file P02_24.xlsx. Comment on any observed trends in the annual educa- tional attainment of the general U.S. population over the given period.

25. The monthly averages of the federal funds rate and the bank prime loan rate are listed in the file P02_25.xlsx. a. Describe the time series behavior of these two variables.

Can you discern any cyclical or other patterns in the times series graphs of these key interest rates?

b. Discuss whether it would make much sense, espe- cially to a person at the present time, to quote tradi- tional summary measures such as means or percentiles of these series.

Level B 26. In which months of the calendar year do U.S. gasoline ser-

vice stations typically have their lowest retail sales levels? In which months of the calendar year do the service sta- tions typically have their highest retail sales levels? Create time series graphs for the monthly data in the file P02_26. xlsx to respond to these two questions. There are really two series, one of actual values and one of seasonally adjusted values. The latter adjusts for any possible sea- sonality, such as higher values in June and lower values in January, so that any trends are more apparent.

Mean Std. Dev. Median Quar�le 1 Quar�le 3

0.67% 4.09% 0.85%

–1.61% 3.22%

16 17 18 19 20 21 22 23 24

D E F G H I J K L M

25 26 27 28 29 30

Summary measures

Ja n-

50 Se

p- 53

M ay

-5 7

Ja n-

61 Se

p- 64

Se p-

M ay

-6 8

M ay

-7 9

M ay

-9 0

M ay

-0 1

M ay

-1 2

Ja n-

20.00% 15.00% 10.00%

5.00% 0.00%

–5.00% –10.00% –15.00% –20.00% –25.00% –30.00%

Percentage Change

Figure 2.23 Summary Measures and Graph of Percentage Changes of the Dow

6 8 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

09953_ch02_ptg01_037-083.indd 68 04/03/19 10:41 PM

2-6 Outliers and Missing Values 6 9

27. The file P02_27.xlsx contains monthly data for total U.S. electronic and appliance store sales. There are really two series, one of actual values and one of season- ally adjusted values. The latter adjusts for any possible seasonality, such as higher values in June and lower val- ues in January, so that any trends are more apparent. a. Is there an observable trend in these data? That is, do

the values of the series tend to increase or decrease over time?

b. Is there a seasonal pattern in these data? If so, what is the seasonal pattern?

28. T h e f i l e P 0 2 _ 2 8 . x l s x c o n t a i n s t o t a l m o n t h l y U.S. retail sales data for a number of years. There are really two series, one of actual sales and one of seasonally

adjusted sales. The latter adjusts for any possible season- ality, such as higher sales in December and lower sales in February, so that any trends are more apparent. a. Create a graph of both time series and comment on

any observable trends, including a possible seasonal pattern, in the data. Does seasonal adjustment make a difference? How?

b. Based on your time series graph of actual sales, make a qualitative projection about the total retail sales levels for the next 12 months. Specifically, in which months of the subsequent year do you expect retail sales levels to be highest? In which months of the subsequent year do you expect retail sales levels to be lowest?

2-6 Outliers and Missing Values Most textbooks on data analysis, including this one, tend to use example data sets that are “cleaned up.” Unfortunately, the data sets you are likely to encounter in your job are often not so clean. Two particular problems you will encounter are outliers and missing data, the topics of this section. There are no easy answers for dealing with these problems, but you should at least be aware of the issues.

Outliers An outlier is literally a value or an entire observation (row) that lies well outside of the norm. For the baseball data, Mike Trout’s salary of $34 million is definitely an outlier. This is indeed his correct salary—the number wasn’t entered incorrectly—but it is way beyond what most players make. Actually, statisticians disagree on an exact definition of an outlier. Going by the third empirical rule, you might define an outlier as any value more than three standard deviations from the mean, but this is only a rule of thumb. Let’s just agree to define outliers as extreme values, and then for any particular data set, you can decide how extreme a value needs to be to qualify as an outlier.

Sometimes an outlier is easy to detect and deal with. For example, this is often the case with data entry errors. Suppose a data set includes a Height variable, a person’s height measured in inches, and you see a value of 720. This is certainly an outlier—and it is cer- tainly an error. Once you spot it, you can go back and check this observation to see what the person’s height should be. Maybe an extra 0 was accidentally appended and the true value is 72. In any case, this type of outlier is usually easy to discover and fix.

Sometimes a careful check of the variable values, one variable at a time, will not reveal any outliers, but there still might be unusual combinations of values. For example, it would be strange to find a person with Age equal to 10 and Height equal to 72. Neither of these values is unusual by itself, but the combination is certainly unusual. Again, this would probably be a result of a data entry error, but it would be harder to spot. (The scatterplots discussed in the next chapter are useful for spotting unusual combinations.)

It isn’t always easy to detect outliers, but an even more important issue is what to do about them when they are detected. Of course, if they are due to data entry errors, they can be fixed, but what if they are legitimate values like Mike Trout’s salary? One or a few wild outliers like this one can dominate a statistical analysis. For example, they can make a mean or standard deviation much different than if the outliers were not present.

For this reason, some people argue, possibly naïvely, that outliers should be elim- inated before running statistical analyses. However, it is not appropriate to eliminate outliers simply to produce “nicer” results. There has to be a legitimate reason for elim- inating outliers, and such a reason sometimes exists. For example, suppose you want to analyze salaries of “typical” managers at your company. Then it is probably appropriate to

09953_ch02_ptg01_037-083.indd 69 04/03/19 10:41 PM

7 0 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

eliminate the CEO and possibly other high-ranking executives from the analysis, arguing that they aren’t really part of the population of interest and would just skew the results. Or if you are interested in the selling prices of “typical” homes in your community, it is prob- ably appropriate to eliminate the few homes that sell for over $2 million, again arguing that these are not the types of homes you are interested in.

Probably the best advice for dealing with outliers is to run the analyses two ways: with the outliers and without them. This way, you can report the results both ways—and you are being honest.

Missing Values There are no missing data in the baseball salary data set. All 877 observations have a value for each of the four variables. For real data sets, however, this is probably the exception rather than the rule. Unfortunately, most real data sets have missing values in the data. This could be because a person didn’t want to provide all the requested personal informa- tion (what business is it of yours how old I am or how much alcohol I drink?), it could be because data doesn’t exist (stock prices in the 1990s for companies that went public after 2000), or it could be because some values are simply unknown. Whatever the reason, you will undoubtedly encounter data sets with varying degrees of missing values.

As with outliers, there are two issues: how to detect missing values and what to do about them. The first issue isn’t as simple as you might imagine. For an Excel data set, you might expect missing data to be obvious from blank cells. This is certainly one possibility, but there are others. Missing data are coded in a variety of strange ways. One common method is to code missing values with an unusual number such as 29999 or 9999. Another method is to code missing values with a symbol such as 2 or *. If you know the code (and it is often supplied in a footnote), then it is usually a good idea, at least in Excel, to perform a global search and replace, replacing all of the missing value codes with blanks.

The more important issue is what to do about missing values. One option is to ignore them. Then you will have to be aware of how the software deals with missing values. For example, if you use Excel’s AVERAGE function on a column of data with missing values, it reacts the way you would hope and expect—it adds all the nonmissing values and divides by the number of nonmissing values. Most statistical software reacts this same way.

Because this is such an important topic in real-world data analysis, researchers have studied many ways of filling in the gaps so that the missing data problem goes away (or is at least disguised). One possibility is to fill in all of the missing values in a column with the average of the nonmissing values in that column. Indeed, this is an option in some software packages, but it is often not a very good option. (Is there any reason to believe that missing values would be average values if they were known? Probably not.) Another possibility is to examine the nonmissing values in the row of any missing value. It is pos- sible that they provide some clues on what the missing value should be. For example, if a person is male, is 55 years old, has an MBA degree from Harvard, and has been a manager at an oil company for 25 years, this should probably help to predict his missing salary. (It probably isn’t below $100,000.) We will not discuss this issue any further here because it is quite complex, and there are no easy answers. But be aware that you will undoubtedly have to deal with missing data at some point in your job, either by ignoring the missing values or by filling in the gaps in some way.

One good way of dealing with outliers is to report results with the outliers and without them.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 29. The file P02_29.xlsx contains monthly percentages of

on-time arrivals at several of the largest U.S. airports and all of the major airlines from 1988 to 2008. The “By

Airline” sheet contains a lot of missing data, presum- ably because some of the airlines were not in existence in 1988 and some went out of business before 2008. The “By Airport” sheet contains missing data only for Atlantic City International Airport (and we’re not sure why). a. Calculate summary measures (means, medians, stan-

dard deviations, and any other measures you would like to report) for each airline and each airport. How do these deal with missing data?

09953_ch02_ptg01_037-083.indd 70 04/03/19 10:41 PM

2-7 excel tables for Filtering, Sorting, and Summarizing 7 1

b. Create histograms for a few of the airports and a few of the airlines, including Atlantic City International. How do these deal with missing data?

c. Create time series graphs for a few of the airports and a few of the airlines, including Atlantic City Interna- tional. How do these deal with missing data?

d. Which airports and which airlines have done a good job? Which would you like to avoid?

30. (Requires Excel 2016 or later.) The Wall Street Journal CEO Compensation Study analyzed CEO pay for many U.S. companies with fiscal year 2008 revenue of at least $5 billion that filed their proxy statements between October 2008 and March 2009. The data are in the file P02_30.xlsx. a. Create a new variable that is the sum of salary and

bonus, and create a box plot of this new variable. b. As the box plot key indicates, mild outliers are obser-

vations between 1.5 IQR (interquartile range) and 3.0 IQR from the edge of the box, whereas extreme outliers are greater than 3 IQR from the edge of the box. Use these definitions to identify the names of all CEOs who are mild outliers and all those who are extreme outliers.

Level B 31. There is no consistent way of defining an outlier that

everyone agrees upon. For example, some people refer to an outlier that is any observation more than three stan- dard deviations from the mean. Other people use the box

plot definition, where an outlier (moderate or extreme) is any observation more than 1.5 IQR from the edges of the box, and some people care only about the extreme box plot-type outliers, those that are 3.0 IQR from the edges of the box. The file P02_18.xlsx contains daily percentage changes in the S&P 500 index over several years. Identify outliers—days when the percentage change was unusually large in either a negative or pos- itive direction—according to each of these three defini- tions. Which definition produces the most outliers?

32. Sometimes it is possible that missing data are predic- tive in the sense that rows with missing data are some- how different from rows without missing data. Check this with the file P02_32.xlsx, which contains blood pressures for 1000 (fictional) people, along with vari- ables that can be related to blood pressure. These other variables have a number of missing values, presumably because the people didn’t want to report certain infor- mation. a. For each of these other variables, find the mean and

standard deviation of blood pressure for all people without missing values and for all people with miss- ing values. Can you conclude that the presence or absence of data for any of these other variables has anything to do with blood pressure?

b. Some analysts suggest filling in missing data for a variable with the mean of the nonmissing values for that variable. Do this for the missing data in the blood pressure data. In general, do you think this is a valid way of filling in missing data? Why or why not?

2-7 Excel Tables for Filtering, Sorting, and Summarizing This section discusses a great tool that was introduced in Excel 2007: tables. It is useful to begin with some terminology and history. Earlier in this chapter, we discussed data arranged in a rectangular range of rows and columns, where each row is an observation and each column is a variable, with variable names at the top of each column. In pre-2007 versions of Excel, data sets of this form were called lists, and Excel provided several tools for dealing with lists. In Excel 2007, recognizing the importance of data sets, Microsoft made them much more prominent and provided better tools for analyzing them. Specif- ically, you now have the ability to designate a rectangular data set as a table and then employ a number of powerful tools for analyzing their data. These tools include filtering, sorting, and summarizing. We illustrate Excel tables in the following example.

EXAMPLE

2.6 HYTEX’S CUSTOMER DATA The file Catalog Marketing.xlsx contains data on 1000 customers of HyTex, a (fictional) direct marketing company, for the current year. A sample of the data appears in Figure 2.24. The definitions of the variables are fairly straightforward, but details about several of them are listed in cell comments in row 1. HyTex wants to find some useful and quick information about its customers by using an Excel table. How can it proceed?

09953_ch02_ptg01_037-083.indd 71 04/03/19 10:41 PM

Objective To illustrate Excel tables for analyzing the HyTex data.

Solution The range A1:O1001 is in the form of a data set—it is a rectangular range bounded by blank rows and columns, where each row is an observation, each column is a variable, and variable names appear in the top row. Therefore, it is a candidate for an Excel table. However, it doesn’t benefit from Excel’s table tools until you actually designate it as a table. To do so, select any cell in the data set, click the Table button in the left part of the Insert ribbon, and accept the default options. Or as a quicker alternative, use the Ctrl+t key combination. In either case, two things happen. First, the data set is designated as a table, it is formatted nicely, and a dropdown arrow appears next to each variable name, as shown in Figure 2.25. Second, a new Table Tools Design ribbon becomes available (see Figure 2.26). This ribbon is available any time the active cell is inside a table. Note that the table has a generic name like Table1 by default, but you can change this to a more descriptive name if you like.

Figure 2.24 HyTex Customer Data

1 2 3 4 5 6 7 8 9

10 11

A B C D E F G H I J K L M N O Person Age Gender Own Home Married Close Salary Children History Catalogs Region State City First Purchase Amount Spent

1 1 0 0 0 1 $16,400 1 1 12 South Florida Orlando 10/23/2011 $218 2 2 0 1 1 0 $108,100 3 3 18 Midwest Illinois Chicago 5/25/2009 $2,632 3 2 1 1 1 1 $97,300 1 NA 12 South Florida Orlando 8/18/2015 $3,048 4 3 1 1 1 1 $26,800 0 1 12 East Ohio Cleveland 12/26/2012 $435 5 1 1 0 0 1 $11,200 0 NA 6 Midwest Illinois Chicago 8/4/2015 $106 6 2 0 0 0 1 $42,800 0 2 12 West Arizona Phoenix 3/4/2013 $759 7 2 0 0 0 1 $34,700 0 NA 18 Midwest Kansas Kansas City 6/11/2015 $1,615 8 3 0 1 1 0 $80,000 0 3 6 West California San Francisco 8/17/2009 $1,985 9 2 1 1 0 1 $60,300 0 NA 24 Midwest Illinois Chicago 5/29/2015 $2,091

10 3 1 1 1 0 $62,300 0 3 24 South Florida Orlando 6/9/2011 $2,644

Figure 2.25 Table with Dropdown Arrows Next to Variable Names

$16,400 $108,100

$97,300 $26,800 $11,200 $42,800 $34,700 $80,000 $60,300 $62,300

$218 $2,632 $3,048

$435 $106 $759

$1,615 $1,985 $2,091 $2,644

1 0 1 1 1 1 1 0 1 0

0 1 1 1 0 0 0 1 0 1

0 1 1 1 0 0 0 1 1 1

0 0 1 1 1 0 0 0 1 1

1 2 3 4 5 6 7 8 9

2 3 4 5 6 7 8 9

10 11

1 2 2 3 1 2 2 3 2 3

1 3

NA 1

NA 2

NA 3

1 3 1 0 0 0 0 0 0 0

12 18 12 12

6 12 18

6 24 24

South Midwest South East Midwest West Midwest West Midwest South

Florida Illinois Florida Ohio Illinois Arizona Kansas California Illinois Florida

Orlando Chicago Orlando Cleveland Chicago Phoenix Kansas City San Francisco Chicago Orlando

10/23/2011 5/25/2009 8/18/2015

12/26/2012 8/4/2015 3/4/2013

6/11/2015 8/17/2009 5/29/2015 6/9/2011

History Catalogs Region State City First Purchase Amount SpentChildrenSalaryCloseMarriedOwn HomeGenderAgePerson1 A B C D E F G H I J K L M N O

Figure 2.26 Table Tools Design Ribbon

One handy feature of Excel tables is that the variable names remain visible even when you scroll down the screen. Try it to see how it works. When you scroll down far enough that the variable names would disappear, the column headers, A, B, C, etc., change to the variable names. Therefore, you no longer need to freeze panes or split the screen to see the variable names. However, this works only when the active cell is within the table. If you click outside the table, the column headers revert back to A, B, C, etc.

The dropdown arrows next to the variable names allow you to filter in many different ways. For example, click the Own Home dropdown list, uncheck Select All, and check the 1 box. This filters out all customers except those who own their own home. Filtering is discussed in more detail later on, but at this point, just be aware that filtering does not delete any observa- tions; it only hides them. There are three indications that the table has been filtered: (1) the row numbers are colored blue and some are missing; (2) a message appears at the bottom of the screen indicating that only 516 out of 1000 records are visible; and (3) there is a filter icon next to the Own Home dropdown arrow. It is easy to remove this filter by opening the Own Home dropdown list and selecting Clear Filter, but don’t do so yet.

7 2 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

You can also designate a table by selecting any option on the Format as Table dropdown on the Home ribbon. A third alternative is even easier: select any cell in the data set and press Ctrl 1 t.

09953_ch02_ptg01_037-083.indd 72 04/03/19 10:41 PM

As illustrated in Figure 2.26, there are various options you can apply to tables, including the following:

• A number of table styles are available for making the table attractive. You can experiment with these, including the various table styles and table style options. Note the dropdown list in the Table Styles group. It gives you many more styles than the ones originally visible. There is even a “no color” style you might prefer.

• In the Tools group, you can click Convert to Range. This undesignates the range as a table (and the dropdown arrows disappear).

• In the Properties group, you can change the name of the table. You can also click the Resize Table button to expand or con- tract the table’s data range.

• A particularly useful option is the Total Row in the Table Style Options group. If you check this, a new row is appended to the bottom of the table (see Figure 2.27). It creates a sum formula in the rightmost column.4 This sum includes only the non- hidden rows. To prove this to yourself, clear the Own Home filter and check the sum. It increases to $1,216,768. This total row is quite flexible. First, you can summarize the last column by a number of summary measures, such as Average, Max, Min, Count, and others. To do so, select cell O1002 and click the dropdown list that appears. Second, you can summarize any other column in the table in the same way. For example, if you select cell G1002, a dropdown list appears for Salary, and you can then summarize Salary with the same summarizing options.

4 The actual formula is 5SUBtOtaL(109,[amountSpent]), where 109 is a code for summing. However, you never need to type any such formula; you can choose the summary function you want from the dropdown list. 5 If you don’t like this type of formula, you can go to Options under the File menu and uncheck the “Use table names in formulas” option in the Formulas group.

Figure 2.27 Total Violent and Property Crimes

$1,081 $1,857

$654 $843

$2,546 $2,464

Person 2 3 2 2 3 3

12 18

6 12 18 24

Midwest Midwest South West East West

Kansas Ohio Florida Washington Pennsylvania Utah

Kansas City Cincinna� Miami Sea�le Philadelphia Salt Lake City

12/26/2011 10/23/2012

7/7/2013 8/14/2015 8/9/2013 3/9/2012

$796,260

993 994 996 997 999

1001 1002

992 993 995 996 998

1000 Total

History 1 0 0 0 0 1

Children 1 1 0 0 1 1

Married 1 1 1 0 1 1

Gender 1 1 1 1 1 1

Own Home $65,900 $59,700 $41,900 $53,800

$102,600 $102,500

Salary 1 0 1 1 1 1

Close 2 3 3 2 2 2

Age Catalogs Region State City First Purchase Amount Spent

Excel tables have a lot of built-in intelligence. Although there is not enough space here to explain all possibilities, try the following to see what we mean:

• In cell R2, enter a formula by typing an equals sign, pointing to cell O2, typing a divide sign (/), and pointing to cell G2. You do not get the usual formula 5O2/G2.

• Instead you get =Table1[[#This Row],[AmountSpent]]/Table1[[#This Row],[Salary]]. This is certainly not the Excel syntax you are used to, but it makes perfect sense.5

• Similarly, you can expand the table with a new variable, such as the ratio of Amount Spent to Salary. Start by typing the variable name Ratio in cell P1. Then in cell P2, enter a formula exactly as you did in the previous bullet. You will notice two things. First, as soon as you enter the Ratio label, column P becomes part of the table. Second, as soon as you enter the new formula in one cell, it is automatically copied to all of column P. This is what we mean by table intelligence.

• Excel tables expand automatically as new rows are added to the bottom or new columns are added to the right. (You saw this latter behavior in the previous bullet.) To appreciate the benefit of this, suppose you have a monthly time series data set. You designate it as a table and then build a line chart from it to show the time series behavior. Later on, if you add new data to the bottom of the table, the chart will automatically update to include the new data. This is a great feature. In fact, when we discuss pivot tables in the next chapter, we will recommend basing them on tables, not ranges, whenever possible. Then they too will update automatically when new data are added to the table.

Filtering We now discuss ways of filtering data sets—that is, finding records that match particular criteria. One way to filter is to create an Excel table, as indicated in the previous subsection. This auto- matically provides the dropdown arrows next to the field names that provide many filter options. Indeed, this is the way we will filter in this section: on an existing table. However, a designated table is not required for filtering. You can filter on any rectangular data set with variable names. There are actually three ways to do so. For each method, the active cell should be a cell inside the data set.

The Total row in an Excel table summarizes only the visible data. The data that has been filtered out is ignored.

Filtering is possible without using Excel tables, but there are definitely advantages to filtering with Excel tables.

2-7 excel tables for Filtering, Sorting, and Summarizing 7 3

09953_ch02_ptg01_037-083.indd 73 04/03/19 10:41 PM

• Use the Filter button from the Sort & Filter dropdown list on the Home ribbon.

• Use the Filter button from the Sort & Filter group on the Data ribbon. • Right-click any cell in the data set and select Filter. You get several options, the most popular of which is Filter by Selected

Cell’s Value. For example, if the selected cell has value 1 and is in the Children column, then only customers with a single child will remain visible.

The point is that Microsoft realizes how important filtering is to Excel users. Therefore, they have made filtering a very prominent and powerful Excel tool.

As far as we can tell, the two main advantages of filtering on a table, as opposed to the three options just listed, are the nice formatting (banded rows, for example) provided by tables, and, more importantly, the total row. If this total row is show- ing, it summarizes only the visible records; the rows hidden by filters are ignored.

We now illustrate a number of filtering possibilities for the HyTex data. We won’t lead you through a lot of descriptions and screenshots. Once you know the possibilities that are available, you should find them quite easy to use.

• Filter on one or more values in a field. Click the Catalogs dropdown arrow. You will see five checkboxes, all checked: Select All, 6, 12, 18, and 24. To select one or more values, uncheck Select All and then check any values you want to filter on, such as 6 and 24. In this case, only customers who received 6 or 24 catalogs will remain visible.

• Filter on more than one field. With the Catalogs filter still in place, create a filter on some other field, such as customers with one child. When there are filters on multiple fields, only records that meet all of the criteria are visible, in this case customers with one child who received 6 or 24 catalogs.

• Filter on a continuous numeric field. The Salary and Amount Spent fields are essentially continuous fields, so it would not make much sense to filter on one or a few particular values. However, it does make sense to filter on ranges of values, such as all salaries greater than $75,000. This is easy. Click the drop- down arrow next to Salary and select Number Filters. You will see a number of obvious possi- bilities, including Greater Than.

• Top 10 and Above/Below Average filters. Continuing the previous bullet, the Number Filters include Top 10, Above Average, and Below Average options. These are particularly useful if you like to see the highs and the lows. The Above Average and Below Average filters do exactly what their names imply. The Top 10 filter is actually more flexible than its name implies. It can be used to select the top n items (where you can choose n), the bottom n items, the top n percent of items, or the bottom n percent of items. Note that if a Top 10 filter is used on a text field, the ordering is alphabetical. If it is used on a date field, the ordering is chronological.

• Filter on a text field. If you click the dropdown arrow for a text field such as Region, you can choose one or more of its values, such as East and South, to filter on. You can also select the Text Filters item, which provides a number of choices, including Begins With, Ends With, Contains, and others. For example, if there were an Address field, you could use the Begins With option to find all addresses that begin with P.O. Box.

• Filter on a date field. Excel has great built-in intelligence for filtering on dates. If you click the First Purchase dropdown arrow, you will see an item for each year in the data set with plus signs next to them. By clicking on the plus signs, you can drill down to months and then days for as much control as you need. Figure 2.28 shows one possibility, where we have filtered out all dates except the first part of July 2015. In addition, if you select the Date Filters item, you get a number of possibilities, such as Yesterday, Next Week, Last Month, and many others.

• Filter on color or icon. Excel has many ways to color cells or enter icons in cells. Often the purpose is to denote the sizes of the numbers in the cells, such as red for small numbers and green for large numbers. We won’t cover the possibilities in this book, but you can experiment with Conditional Formatting on the Home ribbon. The point is that cells are often col- ored in certain ways or contain certain icons. Therefore, Excel allows you to filter on background color, font color, or icon. For example, if certain salaries are colored yellow, you can isolate them by filtering on yellow.

• Use a custom filter. If nothing else works, you can try a custom filter, available at the bottom of the Number Filters, Text Filters, and Date Filters lists. Figures 2.29 and 2.30 illustrate two possibilities. The first of these filters out all salaries between $25,000 and $75,000. Without a custom filter, this wouldn’t be possible. The second filter uses the * wildcard to find regions ending in est (West and Midwest). Admittedly, this is an awkward way to perform this filter, but it indicates how flexible custom filters can be.

We remind you once again that if you filter on an Excel table and you have summary measures in a total row at the bottom of the table, these summary measures are based only on the filtered data; they ignore the hidden rows.

One final comment about filters is that when you click the dropdown arrow for any variable, you always get three items at the top for sorting, not filtering (see Figure 2.28, for example). These allow you to perform the obvious sorts, from high to low

The number of ways you can filter in Excel tables is virtually unlim- ited. Don’t be afraid to experiment. You can always clear filters to get back to where you started.

7 4 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

09953_ch02_ptg01_037-083.indd 74 04/03/19 10:41 PM

Figure 2.28 Filtering on a Date Variable

Figure 2.30 Custom Filter for Region

Figure 2.29 Custom Filter for Salary

2-7 excel tables for Filtering, Sorting, and Summarizing 7 5

09953_ch02_ptg01_037-083.indd 75 04/03/19 10:41 PM

or vice versa, and they even allow you to sort on color. As with filtering, you do not need to designate an Excel table to perform sorting (the popular A-Z and Z-A buttons work just fine without tables), but sorting is made even easier with tables.

Now that you know the possibilities, here is one particular filter you can try. Suppose HyTex wants information about all middle-aged married customers with at least two children who have above average salaries, own their own home, and live in Indiana or Kentucky. You should be able to create this filter in a few seconds. The result, sorted in decreasing order of Amount Spent and shown in Figure 2.31, indicates that the average salary for these 10 customers is $84,750, and their total amount spent at HyTex is $14,709. (We summarized Salary by average and Amount Spent by sum in the total row.)

Other Useful Ways to Filter

The Advanced Filter tool is useful for implementing an OR filter, such as all customers who are either male with salary above $40,000 OR female with at least two children. In fact, it is probably the most natural way to im- plement such a filter. However, there is another way that is used in the solutions of a few problems in Chapter 2. Suppose you want a filter of the type A OR B, where A and B are any conditions, maybe even AND conditions. Create two columns labeled Condition A and Condition B. Fill each with an IF formula that returns Yes or No, depending on whether the condition holds in that row. Then create a third new column labeled Condition A or B, and fill it with a formula like =IF(OR(conditionA, conditionB), “Yes”, “No”), where conditionA and con- ditionB are references to the Yes/No values in the Condition A and Condition B columns. Then you can filter on Yes in this last column to obtain the desired results.

Excel Tip

Figure 2.31 Results from a Typical Filter

History Catalogs Region State City First Purchase Amount SpentChildrenSalaryCloseMarriedOwn HomeGenderAgePerson

7 6 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 33. The file P02_03.xlsx contains data from a survey of 399

people regarding an environmental policy. Use filters for each of the following. a. Identify all respondents who are female, middle-aged,

and have two children. What is the average salary of these respondents?

b. Identify all respondents who are elderly and strongly disagree with the environmental policy. What is the average salary of these respondents?

c. Identify all respondents who strongly agree with the environmental policy. What proportion of these indi- viduals are young?

d. Identify all respondents who are either (1) middle- aged men with at least one child and an annual salary of at least $50,000, or (2) middle-aged women with two or fewer children and an annual salary of at least $30,000. What are the mean and median salaries of the respondents who meet these conditions? What propor- tion of the respondents who satisfy these conditions agree or strongly agree with the environmental policy?

34. The file P02_07.xlsx includes data on 204 employees at the (fictional) company Beta Technologies. Use filters for each of the following. a. Identify all employees who are male and have exactly

4 years of post-secondary education. What is the aver- age salary of these employees?

09953_ch02_ptg01_037-083.indd 76 04/03/19 10:41 PM

2-8 Conclusion 7 7

b. Find the average salary of all female employees who have exactly 4 years of post-secondary educa- tion. How does this mean salary compare to the one obtained in part a?

c. Identify all employees who have more than 4 years of post-secondary education. What proportion of these employees are male?

d. Identify all full-time employees who are either (1) females between the ages of 30 and 50 (inclusive) with at least 5 years of prior work experience, at least 10 years of prior work experience at Beta, and at least 4 years of post-secondary education; or (2) males between the ages of 40 and 60 (inclusive) with at least 6 years of prior work experience, at least 12 years of prior work experience at Beta, and at least 4 years of post-secondary education.

e. For those employees who meet the conditions speci- fied in part d, compare the mean salary of the females with that of the males. Also, compare the median salary of the female employees with that of the male employees.

f. What proportion of the full-time employees identified in part d earns less than $50,000 per year?

35. The file P02_35.xlsx contains data from a survey of 500 randomly selected households. Use Excel filters to answer the following questions. a. What are the average monthly home mortgage pay-

ment, average monthly utility bill, and average total debt (excluding the home mortgage) of all homeown- ers residing in the southeast part of the city?

b. What are the average monthly home mortgage pay- ment, average monthly utility bill, and average total debt (excluding the home mortgage) of all homeown- ers residing in the northwest part of the city? How do these results compare to those found in part a?

c. What is the average annual income of the first house- hold wage earners who rent their home (house or apartment)? How does this compare to the average annual income of the first household wage earners who own their home?

d. What proportion of the surveyed households contains a single person who owns his or her home?

36. Recall that the file Supermarket Transactions.xlsx contains over 14,000 transactions made by super- market customers over a period of approximately two years. Use Excel filters to answer the following questions. a. What proportion of these transactions are made by

customers who are married? b. What proportion of these transactions are made by

customers who do not own a home? c. What proportion of these transactions are made by

customers who have at least one child? d. What proportion of these supermarket customers are

single and own a home?

Level B 37. The file P02_35.xlsx contains data from a survey of

500 randomly selected households. Use Excel filters to answer the following questions. a. Identify households that own their home and have a

monthly home mortgage payment in the top quartile of the monthly payments for all households.

b. Identify households with monthly expenditures on utilities that are within two standard deviations of the mean monthly expenditure on utilities for all households.

c. Identify households with total indebtedness (exclud- ing home mortgage) less than 10% of the household’s primary annual income level.

2-8 Conclusion The summary measures, charts, and tables discussed in this chapter are extremely useful for describing variables in data sets. The methods in this chapter (and the next two chapters) are often called exploratory methods because they allow you to explore the characteristics of the data and at least tentatively answer interesting questions. Most of these tools have been avail- able for many years, but with the powerful software now accessible to virtually everyone, the tools can be applied quickly and easily to gain insights. We can promise that you will be using many if not all of these tools in your job. Indeed, the knowledge you gain from these early chapters is arguably the most valuable knowledge you will gain from the book.

To help you remember which analyses are appropriate for different questions and different data types, and which tools are useful for performing the various analyses, we have created a taxonomy in the file Data Analysis Taxonomy.xlsx. (It isn’t shown here because it doesn’t fit nicely on the printed page.) Feel free to refer back to the diagram in this file as you apply the tools in this chapter and the next chapter.

09953_ch02_ptg01_037-083.indd 77 04/03/19 10:41 PM

7 8 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

Summary of Key Terms TERM EXPLANATION EXCEL PAGES EQUATION population Includes all objects of interest in a study—

people, households, machines, etc. 22

Sample Representative subset of population, usually chosen randomly

Variable (or field or attribute)

Attribute or measurement of members of a population, such as height, gender, or salary

Observation (or case or record)

List of all variable values for a single member of a population

Data set (Usually) a rectangular array of data, with vari- ables in columns, observations in rows, and variable names in the top row

Data types Several categorizations are possible: numeric versus categorical, discrete versus continuous, cross-sectional versus time series; categorical can be nominal or ordinal

Dummy variable A variable coded 1 or 0: 1 for observations in the category, 0 for observations not in the category

Binned (or discretized) variable

Numeric variable that has been categorized into discrete categories called bins

Mean Average of observations AVERAGE 32 2.1

Median Middle observation after sorting MEDIAN 32

Mode Most frequent observation MODE 32

percentiles Values that have specified percentages of observations below them

PERCENTILE 33

Quartiles Values that have 25%, 50%, or 75% of observations below them

QUARTILE 33

Minimum Smallest observation MIN 33

Maximum Largest observation MAX 33

Concatenate String together two or more pieces of text & character (or CONCATENATE function)

range Difference between largest and smallest observations

MAX, MIN 35

Interquartile range (IQr)

Difference between first and third quartiles QUARTILE 35

Variance Measure of variability; essentially the average of squared deviations from the mean

VAR.S (or VAR.P) 35 2.2, 2.3

Standard deviation Measure of variability in same units as observations; square root of variance

STDEV.S (or STDEV.P)

empirical rules Rules that specify approximate percentage observations within one, two, or three standard deviations of mean for symmetric, bell-shaped distributions

Mean absolute Deviation (MaD)

Another measure of variability; average of absolute deviations from the mean

AVEDEV 39 2.4

Skewness When one tail of a distribution is longer than the other

SKEW 39

Kurtosis Measure of “fatness” of tails of a distribution (relative to a normal distribution)

KURT 39

09953_ch02_ptg01_037-083.indd 78 04/03/19 10:41 PM

2-8 Conclusion 7 9

TERM EXPLANATION EXCEL PAGES EQUATION

histogram Chart of bin counts for a numeric variable; shows shape of the distribution

Excel 2016 or later 45

Box plots Alternative chart that shows the distribution of a numeric variable

Excel 2016 or later 45

time series graph Graph showing behavior through time of one or more time series variables

Excel line chart 45

Outlier Observation that lies outside of the general range of observations in a data set

Missing values Values that are not reported in a data set 62

excel table Rectangular range specified as a table; especially useful for sorting, filtering, and summarizing

Table from Insert ribbon (or Ctrl+t)

Problems

Conceptual Questions C.1. An airline analyst wishes to estimate the proportion of

all American adults who are afraid to fly because of potential terrorist attacks. To estimate this percentage, the analyst decides to survey 1500 Americans from across the nation. Identify the relevant sample and population in this situation.

C.2. The number of children living in each of a large num- ber of randomly selected households is an example of which data type? Be specific.

C.3. Does it make sense to construct a histogram for the state of residence of randomly selected individuals in a sample? Explain why or why not.

C.4. Characterize the likely shape of a histogram of the dis- tribution of scores on a midterm exam in a graduate statistics course.

C.5. A researcher is interested in determining whether there is a relationship between the weekly number of room air-conditioning units sold and the time of year. What type of descriptive chart would be most useful in per- forming this analysis? Explain your choice.

C.6. Suppose that the histogram of a given income distri- bution is positively skewed. What does this fact imply about the relationship between the mean and median of this distribution?

C.7. “The midpoint of the line segment joining the first quar- tile and third quartile of any distribution is the median.” Is this statement true or false? Explain your answer.

C.8. Explain why the standard deviation would likely not be a reliable measure of variability for a distribution of data that includes at least one extreme outlier.

C.9. Explain how a box plot can be used to determine whether a distribution of values is essentially symmetric.

C.10. Suppose that you collect a random sample of 250 salaries for the salespersons employed by a large PC manufacturer. Furthermore, assume that you find that two of these salaries are considerably higher than the

others in the sample. Before analyzing this data set, should you delete the unusual observations? Explain why or why not.

Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 38. The file P02_35.xlsx contains data from a survey of 500

randomly selected households. a. Indicate the type of data for each of the variables

included in the survey. b. For each of the categorical variables in the survey, indi-

cate whether the variable is nominal or ordinal, and why. c. Create a histogram for each of the numeric variables

d. Find the maximum and minimum debt levels for the households in this sample.

e. Find the indebtedness levels at each of the 25th, 50th, and 75th percentiles.

f. Find and interpret the interquartile range for the indebtedness levels of these households.

39. The file P02_39.xlsx contains SAT and ACT test scores by state for high school graduates of the 2016–2017 school year. These are broken down by reading/writing and math/science scores. The file also lists the percent- ages of students taking these two tests by state. a. Create a histogram for each of the numeric variables.

Are these distributions essentially symmetric or are they skewed?

b. Compare the two “Pct taking” columns. Does any- thing stand out?

c. Find summary statistics for each of the variables. Which is the most appropriate measure of central tendency? Why?

d. For the SAT, the Total score is the sum of the ERW and Math scores. How does the mean of the Total

09953_ch02_ptg01_037-083.indd 79 04/03/19 10:41 PM

8 0 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

variable relate to the means of the ERW and Math variables? Is the same true for medians?

40. The Wall Street Journal CEO Compensation Study ana- lyzed CEO pay from many U.S. companies with fiscal year 2008 revenue of at least $5 billion that filed their proxy statements between October 2008 and March 2009. The data are in the file P02_30.xlsx. a. Create histograms to gain a clearer understanding of

the distributions of annual base salaries and bonuses earned by the surveyed CEOs in fiscal 2008. How would you characterize these histograms?

b. Find the annual salary below which 75% of all given CEO salaries fall.

c. Find the annual bonus above which 55% of all given CEO bonuses fall.

d. Determine the range of the middle 50% of all given total direct compensation figures. For the 50% of the executives that do not fall into this middle 50% range, is there more variability in total direct compen- sation to the right than to the left? Explain.

41. The file P02_41.xlsx contains monthly returns on American Express stock for several years. As the formulas in the file indicate, each return is the percentage change in the adjusted closing price from one month to the next. Do monthly stock returns appear to be skewed or symmetric? On average, do they tend to be positive, negative, or zero?

42. The file P02_42.xlsx contains monthly returns on Mattel stock for several years. As the formulas in the file indicate, each return is the percentage change in the adjusted clos- ing price from one month to the next. Create a histogram of these returns and summarize what you learn from it. On average, do the returns tend to be positive, negative, or zero?

43. The file P02_43.xlsx contains U.S. Bureau of Labor Statistics data on the monthly unemployment percent- age among people 25 years old or older, broken down by education level. Use box plots and time series graphs to compare the unemployment levels in the different edu- cation groups. What are the advantages of box plots? What are the advantages of time series graphs?

44. The file P02_44.xlsx contains annual data on the per- centage of people in the United States living below the poverty level. a. In which years of the sample has the poverty rate

exceeded the rate that defines the third quartile of these data?

b. In which years of the sample has the poverty rate fallen below the rate that defines the first quartile of these data?

c. What is the typical poverty rate during this period? d. Create and interpret a time series graph for these data.

How successful have Americans been recently in their efforts to win “the war against poverty”?

e. Given that this data set is a time series, discuss whether the measures requested in parts a-c are very meaningful at the current time.

Level B 45. The file P02_45.xlsx contains the salaries of 135 busi-

ness school professors at a (fictional) large state univer- sity. a. If you increased every professor’s salary by $1000,

what would happen to the mean and median salary? b. If you increased every professor’s salary by $1000,

what would happen to the standard deviation of the salaries?

c. If you increased every professor’s salary by 5%, what would happen to the standard deviation of the salaries?

46. The file P02_46.xlsx lists the fraction of U.S. men and women of various heights and weights. Use these data to estimate the mean and standard deviation of the height of American men and women. (Hint: Assume all heights in a group are concentrated at the group’s midpoint.) Do the same for weights.

47. Recall that HyTex Company is a direct marketer of tech- nical products and that the file Catalog Marketing.xlsx contains recent data on 1000 HyTex customers. a. Identify all customers in the data set who are 55 years

of age or younger, female, single, and who have had at least some dealings with HyTex before this year. Find the average number of catalogs sent to these custom- ers and the average amount spent by these customers this year.

b. Do any of the customers who satisfy the conditions stated in part a have salaries that fall in the bottom 10% of all 1000 combined salaries in the data set? If so, how many?

c. Identify all customers in the sample who are more than 30 years of age, male, homeowners, married, and who have had little if any dealings with HyTex before this year. Find the average combined household sal- ary and the average amount spent by these customers this year.

d. Do any of the customers who satisfy the conditions stated in part c have salaries that fall in the top 10% of all 1000 combined salaries in the data set? If so, how many?

48. Recall that the file Baseball Salaries.xlsx contains data on 877 MLB players in the 2018 season. Using this data set, answer the following questions: a. Find the mean and median of the salaries of all short-

stops. Are any of these measures influenced signifi- cantly by one or more unusual observations?

b. Find the standard deviation, first and third quar- tiles, and 5th and 95th percentiles for the salaries of all shortstops. Are any of these measures influenced significantly by one or more unusual observations?

c. Create a histogram of the salaries of all shortstops. Are any of these measures influenced significantly by one or more unusual observations?

09953_ch02_ptg01_037-083.indd 80 04/03/19 10:41 PM

2-8 Conclusion 8 1

49. (Requires Excel 2016 or later) In 1969 and again in 1970, a lottery was held to determine who would be drafted and sent to Vietnam in the following year. For each date of the year, a ball was put into an urn. For example, in the first lottery, January 1 was number 305 and February 14 was number 4. This means that a person born on February 14 would be drafted before a person born on January 1. The file P02_49.xlsx contains the “draft number” for each date for the two lotteries. Do you notice anything unusual about the results of either lottery? What do you think might have caused this result? (Hint: Create a box plot for each month’s numbers.)

50. The file P02_50.xlsx contains the average price of gas- oline in each of the 50 states for each of several years. (Note: You will need to manipulate the data to some extent before performing the analyses requested below.) a. Compare the distributions of gasoline price data (one

for each year) across states. Specifically, did the mean and standard deviation of these distributions change over time? If so, how do you explain the trends?

b. In which regions of the country did gasoline prices change the most?

c. In which regions of the country did gasoline prices remain relatively stable?

51. The file P02_51.xlsx contains data on U.S. home- ownership rates by state for each of several years. a. Employ numeric summary measures to characterize

the changes in home ownership rates across the coun- try during this period.

b. Do the trends appear to be uniform across the United States or are they unique to certain regions of the country? Explain.

52. Recall that HyTex Company is a direct marketer of tech- nical products and that the file Catalog Marketing.xlsx contains recent data on 1000 HyTex customers. a. Identify all customers who are either (1) home-owners

between the ages of 31 and 55 who live reasonably close to a shopping area that sells similar merchan- dise, and have a combined salary between $40,000 and $90,000 (inclusive) and a history of being a medium or high spender at HyTex; or (2) homeowners greater than the age of 55 who live reasonably close to a shopping area that sells similar merchandise and have a combined salary between $40,000 and $90,000 (inclusive) and a history of being a medium or high spender at HyTex.

b. Characterize the subset of customers who satisfy the conditions specified in part a. In particular, what pro- portion of these customers are women? What propor- tion of these customers are married? On average, how many children do these customers have? Finally, how many catalogs do these customers typically receive, and how much do they typically spend each year at HyTex?

c. In what ways are the customers who satisfy condition 1 in part a different from those who satisfy condition 2 in part a? Be specific.

53. Recall that the file Supermarket Transactions.xlsx contains data on over 14,000 transactions. There are two numeric variables, Units Sold and Revenue. The first of these is discrete and the second is continuous. For each of the following, do whatever it takes to create a bar chart of counts for Units Sold and a histogram of Revenue for the given subpopulation of purchases. a. All purchases made during January and February of

2016. b. All purchase made by married female homeowners. c. All purchases made in the state of California. d. A l l p u r c h a s e s m a d e i n t h e P r o d u c e p r o d u c t

department. 54. The file P02_54.xlsx contains daily values of an EPA air

quality index in Washington DC and Los Angeles from January 1980 through April 2009. For some unknown reason, the source provides slightly different dates for the two cities. a. Starting in column G, create three new columns: Date,

Wash DC Index, and LA Index. Fill the new date column with all dates from 1/1/1980 to 4/30/2009. Then use lookup functions to fill in the two new index columns, entering the observed index if available or a blank otherwise. (Hint: Use a combination of the VLOOKUP function with False as the last argument and the IFERROR function. Look up the latter in online help if you have never seen it before.)

b. Create a separate time series graph of each new index column. Because there are so many dates, it is diffi- cult to see how the graph deals with missing data, but see if you can determine this (maybe by expanding the size of the graph or trying a smaller example). In spite of the few missing points, explain the patterns in the graphs and how Washington DC compares to Los Angeles.

55. The file P02_55.xlsx contains monthly sales (in millions of dollars) of beer, wine, and liquor. The data have not been seasonally adjusted, so there might be seasonal patterns that can be discovered. For any month in any year, define that month’s seasonal index as the ratio of its sales value to the average sales value over all months of that year. a. Calculate these seasonal indexes, one for each month

in the series. Do you see a consistent pattern from year to year? If so, what is it?

b. To “deseasonalize” the data and get the season- ally adjusted series often reported, divide each monthly sales value by the corresponding seasonal index from part a. Then create a time series graph of both series, the actual sales and the seasonally adjusted sales. Explain how they are different and why the seasonally adjusted series might be of interest.

56. The file P02_56.xlsx contains monthly values of indexes that measure the amount of energy necessary to heat or cool buildings due to outside temperatures. (See the explanation in the Source sheet of the file.)

09953_ch02_ptg01_037-083.indd 81 04/03/19 10:41 PM

8 2 C h a p t e r 2 D e s c r i b i n g t h e D i s t r i b u t i o n o f a V a r i a b l e

These are reported for each state in the United States and also for several regions, as listed in the Locations sheet, from 1931 to 2000. Create summary measures and/or charts to see whether there is any indication of temperature changes (global warming?) through time, and report your findings.

57. The file P02_57.xlsx contains data on mortgage loans in 2008 for each state in the United States. The file is different from similar ones in this chapter in that each state has its own sheet with the same data laid out in the same format. Each state sheet breaks down all mortgage applications by loan purpose, applicant race, loan type, outcome, and denial reason (for those that were denied).

The question is how a single data set for all states can be created for analysis. The Typical Data Set sheet indi- cates a simple way of doing this, using the powerful but little-known INDIRECT function. This sheet is basi- cally a template for bringing in any pieces of data from the state sheet you would like to examine. a. Create histograms and summary measures for the

example data given in the Typical Data Set sheet and write a short report on your findings.

b. Create a copy of the Typical Data Set sheet and repeat part a on this copy for at least one other set of vari- ables (of your choice) from the state sheets.

CASE 2.1 Correct Interpretation of Means A mean, as defined in this chapter, is a simple concept—it is the average of a set of numbers. But even this simple concept can cause confusion if you aren’t careful. The data in Figure 2.32 are typical of data presented by marketing researchers for a type of product, in this case beer. (See the file C02_01.xlsx.)

Each value is an average of the number of sixpacks of beer purchased per customer during a month. For the indi- vidual brands, the value is the average only for the custom- ers who purchased at least one six-pack of that brand. For example, the value for Miller is the average number of six- packs purchased of all of these brands for customers who purchased at least one six-pack of Miller. In contrast, the “Any” average is the average number of six-packs purchased of these brands for all customers in the population.

Is there a paradox in these averages? On first glance, it might appear unusual, or even impossible, that the “Any” average is less than each brand average. Make up your own (small) data set, where you list a number of customers, along with the number of six-packs of each brand of beer each cus- tomer purchased, and calculate the averages for your data that correspond to those in Figure 2.32. Do you get the same result (that the “Any” average is lower than all of the others)? Are you guaranteed to get this result? Does it depend on the amount of brand loyalty in your population, where brand loyalty is greater when customers tend to stick to the same brand, rather than buying multiple brands? Write up your results in a concise report.

CASE 2.2 The Dow Jones Industrial Average The monthly closing values of the Dow Jones Industrial Average (DJIA) for the period beginning in January 1950 are given in the file C02_02.xlsx. According to Wikipedia, the Dow Jones Industrial Average, also referred to as the Industrial Average, the Dow Jones, the Dow 30, or simply the Dow, is one of several stock market indices created by Charles Dow. The average is named after Dow and one of his business associates, statistician Edward Jones. It is an index that shows how 30 large, publicly owned companies based in the United States have traded during a standard trading

session in the stock market. It is the second oldest U.S. mar- ket index after the Dow Jones Transportation Average, which Dow also created.

The Industrial portion of the name is largely historical, as many of the modern 30 components have little or nothing to do with traditional heavy industry. The average is price- weighted, and to compensate for the effects of stock splits and other adjustments, it is currently a scaled average. The value of the Dow is not the actual average of the prices of its component stocks, but rather the sum of the component

1 2

A B C D E F G H Miller

6.77 6.66 6.64 7.11 7.29 7.3 7.17 4.71 Budweiser Coors Michelob Heineken Old Milwaukee Rolling Rock Any

Figure 2.32 Average Beer Purchases

09953_ch02_ptg01_037-083.indd 82 04/03/19 10:41 PM

2-8 Conclusion 8 3

CASE 2.3 Home and Condo Prices The file C02_03.xlsx contains an index of home prices and a seasonally adjusted (SA) version of this index for sev- eral large U.S. cities. It also contains a condo price index for several large cities and a national index. (The data are explained in the Source sheet.) Use the tools in this chap- ter to explore these data, and write a report of your findings.

Some important questions you can answer are the follow- ing: Are there trends over time? Are there differences across cities? Are there differences across different months of the year? Do condo prices mirror home prices? Why are season- ally adjusted (SA) indexes published?

prices divided by a divisor, which changes whenever one of the component stocks has a stock split or stock dividend, so as to generate a consistent value for the index.

Along with the NASDAQ Composite, the S&P 500 Index, and the Russell 2000 Index, the Dow is among the most closely watched benchmark indices for tracking stock market activity. Although Dow compiled the index to gauge the performance of the industrial sector within the U.S. econ- omy, the index’s performance continues to be influenced not

only by corporate and economic reports, but also by domes- tic and foreign political events such as war and terrorism, as well as by natural disasters that could potentially lead to eco- nomic harm.

Using the summary measures and graphical tools from this chapter, analyze this important time series over the given period. Summarize in detail the behavior of the monthly closing values of the Dow and the associated monthly per- centage changes in the closing values of the Dow.

APPENDIX Introduction to StatTools StatTools is a statistical add-in developed by Palisade as part of their DecisionTools Suite.6 StatTools contains many procedures discussed in this chapter and later chapters of the book. Most of these procedures, including the summary stats, histograms, box plots, and time series graph procedures discussed in this chapter, can be implemented with Excel’s built-in tools, but StatTools can perform them more quickly. Later chapter will make further comments about StatTools procedures relevant to those chapters.

Rather than fill this appendix with a lot of explanations and screenshots, we created an Introduction to StatTools video. This video shows the StatTools ribbon, so that you can see the types of procedures StatTools can perform. StatTools is very easy to learn once you know the basics explained in this video. Therefore, if you decide to use StatTools for the statistical procedures discussed in this chapter and later chapters, we strongly recommend you watch this video first.

6 StatTools was originally based on StatPro, an add-in developed by Albright in the 1990s. A free version of StatPro is still available at https://kelley.iu.edu/albrightbooks/Free_downloads.htm, but Albright no longer supports it.

09953_ch02_ptg01_037-083.indd 83 04/03/19 10:41 PM

CHAPTER 3 Finding Relationships among Variables

DATA ANALYTICS AT NYPD If your impression of the New York Police Department (NYPD) is based on the hit TV show NYPD Blue (1993 to 2005), think again. The NYPD has a history of apply- ing analytics to make its 36,000 uniformed officers more effective and efficient at their jobs. As described in Levine et al. (2017), the NYPD developed CompStat in 1993, a structured process for examining crime statistics. Then on the heels of the terrorism attack on September 11, 2001, they created a Counterterrorism Bureau, the first of its kind in a U.S. municipal police department. Their latest use of analytics, begun in 2008, is the Domain Awareness System (DAS), a network of sensors, databases, devices, software, and infrastructure that delivers up-to-date information and analytics to mobile devices and precinct desktops. This sys-

tem is now deployed across every police precinct in the city, on all officers’ smartphones, and on the tablets in all police vehicles. It provides officers with the timely information they need to respond quickly and appropriately to situations as they arise.

Before DAS, the NYPD had a tremendous amount of information about crimes and offenders, but it didn’t have the means to get this information to the officers who needed it in a timely manner. For example, suppose a 911 call about a domestic disturbance was received. Before DAS, the caller spoke to an operator, whose transcription of the call was passed to a dispatcher. The dispatcher then assigned the job to officers in the vicinity, telling them there was a domestic incident at a stated address. With this limited information, the officers had to manage the situation as well as possible, often having to make life-or-death decisions without any historical context. Imagine how much better the officers could react if they knew ahead of time that there was a warrant for the arrest of the person at this address or that this was the fifth such call from this address in the past few months. With DAS, this type of information is immediately available from one of the officer’s phones. (The officer in the passenger seat looks up the information on the phone; the officer driving is instructed to keep his eyes on the road.) In fact, the officers can now access the caller’s statements as transcribed by the operator, information they never had before DAS.

DAS also implements predictive analytics for more strategic decisions, such as where a precinct commanding officer should place units on patrol. Before DAS, this decision was made by marking locations of crimes in the past month on a map, circling “hot spots” on the map where most crimes seemed to occur, and positioning units at the hot spots. Pre- dictive analytics can improve on this process by predicting where future crimes are likely to occur.

Prior to DAS, there was a weekly CompStat meeting about strategic decisions that revolved around a static piece of paper listing the numbers of crimes of various types for the week, the last 28 days, and year-to-date. There was some summarization on this sheet, such as how this week’s numbers compared to the numbers from a year ago, but not very much. There was almost no way for those at the meeting to use these numbers for strate- gic decisions because they couldn’t see the patterns that led to the numbers. This is now

B u m

b le

D e e /S

h u tt e rs

to c k.

c o m

09953_ch03_ptg01_084-131.indd 84 04/03/19 12:10 PM

3-1 Introduction 8 5

possible with DAS. Each number is now presented in an interactive form on a computer screen. If a user clicks on a number, DAS shows all the records included in that number and marks them on a map. Data visualization software then enables the user to explore trends and patterns with bar charts or time series charts, for example. This information, formerly available only to headquarters staff, is now available to commanding officers and their crime analysts.

The Data system has created a “big data” problem for the NYPD. Audio gunshot detectors, 911 calls, license plate readers (LPRs), closed-circuit television cameras, and environmental sensors all feed real-time data into DAS. As of 2016, it also con- tains two billion readings from license plates (with photos), 54 million 911 calls, 15 million complaints, 12 million detective reports, 11 million arrests, 2 million war- rants, and 30 days of video from 9000 cameras. How can all this data be analyzed? Much of the analysis is based on pattern recognition. For example, pattern recogni- tion is implemented on the LPR data. Predictive analytics inform officers where and when a specific license plate is likely to be scanned in the next few minutes. (This is performed only on vehicles on an NYPD watch list.) Time-and-place patterns occur when a vehicle passes the same LPR multiple times on a specific day of the week and a specific time of day, such as every Tuesday at 2 PM. Routing patterns occur when a vehicle passes a fixed LPR and then later passes a second fixed LPR, such as enter- ing the Queensboro Bridge and then driving south on FDR Drive, and it does this on multiple days. These types of patterns give officers a better chance of finding suspected vehicles by focusing their searches.

The DAS system has been a huge success. NYPD officers tend to be skeptical of new systems involving technology, but DAS was designed for cops by cops. Therefore, they appear to trust it and use it. As a measure of its popularity with NYPD officers, there were 100,000 mobile job clicks per week by the end of 2015.

3-1 Introduction The previous chapter introduced a number of summary measures, graphs, and tables to describe the distribution of a single variable. For a variable such as baseball salary, the entire focus was on how salaries were distributed over some range. This is an important first step in any exploratory data analysis—to look closely at variables one at a time—but it is almost never the last step. The primary interest is usually in relationships between variables. For example, it is natural to ask what drives baseball salaries. Does it depend on qualitative factors, such as the player’s team or position? Does it depend on quantitative factors, such as the number of hits the player gets or the number of strikeouts? To answer these questions, you have to examine relationships between various variables and salary.

This chapter again discusses numerical summary measures, graphs, and tables, but they now involve at least two variables at a time. The most useful numeric summary measure is correlation, a measure that applies primarily to numeric variables. The most useful graph is a scatterplot, which again applies primarily to numeric variables. Other tools are used for relationships involving categorical variables. For example, to break down a numeric variable by a categorical variable, it is often useful to create side-by- side box plots. Finally, we discuss Excel®’s arguably most powerful tool, the pivot table. A pivot table enables you to break down one variable by others so that relationships can be uncovered quickly.

As you read this chapter, remember that the diagram in the file Data Analysis Taxonomy.xlsx is available. This diagram gives you the big picture of which analyses are appropriate for which data types and which tools are best for performing the various analyses.

A key issue in this chapter is that different tools should be used to examine relation- ships, depending on whether the variables involved are numeric or categorical.

09953_ch03_ptg01_084-131.indd 85 04/03/19 12:10 PM

8 6 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

3-2 Relationships among Categorical Variables Consider a data set with at least two categorical variables, Smoking and Drinking. Each person is categorized into one of three smoking categories: nonsmoker (NS), occa- sional smoker (OS), and heavy smoker (HS). Similarly, each person is categorized into one of three drinking categories: nondrinker (ND), occasional drinker (OD), and heavy drinker (HD). Do the data indicate that smoking and drinking habits are related? For exam- ple, do nondrinkers tend to be nonsmokers? Do heavy smokers tend to be heavy drinkers?

As discussed in the previous chapter, the most meaningful way to describe a categori- cal variable is with counts, possibly expressed as percentages of totals, and corresponding charts of the counts. The same is true of examining relationships between two categorical variables. You can find the counts of the categories of either variable separately, and more importantly, you can find counts of the joint categories of the two variables, such as the count of all nondrinkers who are also nonsmokers. Again, corresponding percentages of totals and charts help tell the story.

It is customary to display all such counts in a table called a crosstabs (for cross- tabulations). This is also sometimes called a contingency table. Example 3.1 illustrates these tables.

Use a crosstabs, a table of counts of joint categories, to discover relationships between two categorical variables.

EXAMPLE

3.1 RELATIONSHIP BETWEEN SMOKING AND DRINKING The file Smoking Drinking.xlsx lists the smoking and drinking habits of 8761 adults. (This is fictional data.) The cate- gories have been coded so that “N,” “O,” and “H” stand for “Non,” “Occasional,” and “Heavy,” and “S” and “D” stand for “Smoker” and “Drinker.” Is there any indication that smoking and drinking habits are related? If so, how are they related?

Objective To use a crosstabs to explore the relationship between smoking and drinking.

Solution The first question is the data format. If you are lucky, you will be given a table of counts. However, it is also possible that you will have to create these counts. In the file for this example, the data are in long columns, part of which is shown in Figure 3.1. Presumably, there could be other variables describing these people, but only the Smoking and Drinking variables are relevant here.

1 2 3 4 5 6 7 8 9

10 11

A B C Person Smoking Drinking

1 N S OD 2 N S HD 3 O S HD 4 H S ND 5 N S OD 6 N S ND 7 N S OD 8 N S ND 9 O S HD

1 0 H S HD

Figure 3.1 Smoking and Drinking Data

09953_ch03_ptg01_084-131.indd 86 04/03/19 12:10 PM

Figure 3.2 Crosstabs of Smoking and Drinking

1 2 3 4 5

E F G H I Crosstabs from COUNTIFS formulas

NS OS HS Total ND 2118 435 163 2716 OD 2061 1067 552 3680

6 7 8 9

10 11 12

HD 733 899 733 2365 Total 4912 2401 1448 8761

NS OS HS Total ND 78.0 % 16.0 % 6.0% 100.0% OD 56.0 % 29.0 % 15.0 % 100.0%

13 14 15 16 17 18 19

HD 31.0 % 38.0 % 31.0 % 100.0%

NS OS HS ND 43.1 % 18.1 % 11.3% OD 42.0 % 44.4 % 38.1% HD 14.9% 37.4% 50.6%

20 Total 100.0 % 100.0 % 100.0%

Shown as percentages of row totals

Shown as percentages of column totals

To create the crosstabs, start by entering the category headings in row 3 and column E of Figure 3.2. The goal is to fill the table in rows 4–6 with counts of joint categories, along with row and column sums. If you are thinking about using the COUNTIF function to obtain the joint counts, you are close. However, the COUNTIF function lets you specify only a single criterion, and there are now two criteria, one for smoking and one for drinking. Fortunately, Excel has a function designed exactly for this: COUNTIFS. It enables you to specify any number of range-criterion pairs. In fact, you can fill the entire table with a single formula entered in cell F4 and copied to the range F4:H6:

=COUNTIFS($B$2:$B$8762,F$3,$C$2:$C$8762,$E4)

The first two arguments are for the condition on smoking; the last two are for the condition on drinking. You can then sum across rows and down columns to get the totals.

The resulting counts appear in the top table in Figure 3.2. For example, among the 8761 people, 4912 are nonsmokers, 2365 are heavy drinkers, and 733 are nonsmokers and heavy drinkers. Because the totals are far from equal (there are many more nonsmokers than heavy smokers, for example), any relationship between smoking and drinking is difficult to detect in these raw counts. Therefore, it is useful to express the counts as percentages of row totals in the middle table and as percent- ages of column totals in the bottom table.

The latter two tables indicate, in complementary ways, that there is definitely a relationship between smoking and drinking. If there were no relationship, the rows in the middle table would be practically identical, as would the columns in the bottom table. (Make sure you understand why this is true.) But they are far from iden- tical. For example, the middle table indicates that only 6% of the nondrinkers are heavy smokers, whereas 31% of the heavy drinkers are heavy smokers. Similarly, the bottom table indicates that 43.1% of the nonsmokers are nondrinkers, whereas only 11.3% of the heavy smokers are nondrinkers. In short, these tables indicate that smoking and drinking habits tend to go with one another. These tendencies are reinforced by the column charts of the two percentage tables in Figure 3.3.

Relationships between the two variables are usually more evident when the counts are expressed as percentages of row totals or percentages of column totals.

3-2 relationships among Categorical Variables 8 7

09953_ch03_ptg01_084-131.indd 87 04/03/19 12:10 PM

Figure 3.3 Column Charts of Smoking and Drinking Percentages

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

ND OD HD

Percentages of column totals

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

NS OS HS

Percentages of row totals

ND OD HD NS OS HS

Counts Versus Percentages

There is no single correct way to display the data in a crosstabs. Ultimately, the data are always counts, but they can be shown as raw counts, percentages of row totals, percentages of column totals, or even percentages of the overall total. Nev- ertheless, when you are looking for relationships between two categorical vari- ables, showing the counts as percentages of row totals or percentages of column totals usually makes any relationships emerge more clearly. Corresponding charts are also very useful.

Fundamental Insight

Creating Charts from Crosstabs

It takes almost no work to create these charts. To get the one on the left, high- light the range E10:H13 and insert a column chart from the Insert ribbon. Do the same with the range E16:H19 to get the chart on the right, except that it will have smoking on the horizontal axis and drinking in the legend. To reverse their roles, click the Switch Row/Column button on the Chart Tools Design ribbon.

Excel Tip

Although this example illustrates that it doesn’t take too much work to create cross- tabs and corresponding charts, you will see a much quicker and easier way when pivot tables are discussed later in this chapter.

a. For each year separately, recode Nationality so that all nationalities with a count of 1 or 2 are listed as Other.

b. For each year, create a crosstabs of Gender versus the recoded Nationality and an associated column chart. Does there seem to be any relationship between Gender and the recoded Nationality? Is the pattern about the same in the two years?

2. The file P02_03.xlsx contains data from a survey of 399 people regarding a government environmental policy.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. The file P02_01.xlsx indicates the gender and national-

ity of the MBA incoming class in two successive years at the Kelley School of Business at Indiana University.

8 8 C h a P t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 88 04/03/19 8:35 PM

3-3 relationships among Categorical Variables and a Numeric Variable 8 9

Level B 5. Recall from Chapter 2 that HyTex Company is a direct

marketer of technical products and that the file Catalog Marketing.xlsx contains recent data on 1000 HyTex customers. To understand these customers, first recode Salary and Amount Spent as indicated in part a, and then create each of the requested crosstabs and an asso- ciated column chart in parts b to e. Express each count as a percentage, so that for any value of the first variable listed, the percentages add up to 100%. Do any patterns stand out? a. Find the first, second, and third quartiles of Salary,

and then recode Salary as 1 to 4, depending on which quarter of the data each value falls into. For exam- ple, the first salary, $16,400, is recoded as 1 because $16,400 is less than the first quartile, $29,975. Recode Amount Spent similarly, based on its quar- tiles. (Hint: The recoding can be done most easily with lookup tables.)

b. Age versus the recoded Amount Spent c. Own Home versus the recoded Amount Spent d. History versus the recoded Amount Spent e. The recoded Salary versus the recoded Amount Spent

6. The smoking/drinking example in this section used the function COUNTIFS function to find the counts of the joint categories. Without using this function (or pivot tables), devise another way to get the counts. The raw data are in the file Smoking Drinking.xlsx. (Hint: One possibility is to concatenate the values in columns B and C into a new column D. But feel free to find the counts in any way you like.)

a. Create a crosstabs and an associated column chart for Gender versus Opinion. Express the counts as per- centages so that for either gender, the percentages add to 100%. Discuss your findings. Specifically, do the two genders tend to differ in their opinions about the environmental policy?

b. Repeat part a with Age versus Opinion. c. Recode Salary to be categorical with categories “Less

than $40K,” “Between $40K and $70K,” “Between $70K and $100K,” and “Greater than $100K” (where you can treat the breakpoints however you like). Then repeat part a with this new Salary variable versus Opinion.

3. The file P02_02.xlsx contains data on 256 movies that grossed at least $1 million in 2017. a. Recode Distributor so that all distributors with fewer

than 10 movies are listed as Other. Similarly, recode Genre so that all genres with fewer than 10 movies are listed as Other.

b. Create a crosstabs and an associated column chart for these two recoded variables. Express the counts as percentages so that for any distributor, the percentages add to 100%. Discuss your findings.

4. Recall from Chapter 2 that the file Supermarket Trans- actions.xlsx contains over 14,000 transactions made by supermarket customers over a period of approximately two years. To understand which customers purchase which products, create a crosstabs and an associated column chart for each of the following. For each, express the counts as percentages so that for any value of the first variable listed, the percentages add to 100%. Do any patterns stand out? a. Gender versus Product Department b. Marital Status versus Product Department c. Annual Income versus Product Department

3-3 Relationships among Categorical Variables and a Numeric Variable

This section describes a very common situation where the goal is to break down a numeric variable such as salary by a categorical variable such as gender. This is precisely what pivot tables were built for, as you will see later in the chapter. For now, however, Excel’s numeric and graphical tools will be used. This general problem, typically referred to as the comparison problem, is one of the most important problems in data analysis. It occurs whenever you want to compare a numeric variable across two or more subpopulations. Here are some examples:

• The subpopulations are males and females, and the numeric variable is salary. • The subpopulations are different regions of the country, and the numeric variable is the

cost of living. • The subpopulations are different days of the week, and the numeric variable is the

number of customers going to a particular fast-food chain. • The subpopulations are different machines in a manufacturing plant, and the numeric

variable is the number of defective parts produced per day. • The subpopulations are patients who have taken a new drug and those who have taken a

placebo, and the numeric variable is the recovery rate from a particular disease. • The subpopulations are undergraduates with various majors (business, English, history,

and so on), and the numeric variable is the starting salary after graduating.

The comparison problem, where a numeric variable is compared across two or more subpopulations, is one of the most important prob- lems faced by data analysts in all fields of study.

09953_ch03_ptg01_084-131.indd 89 04/03/19 12:10 PM

9 0 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

The list could continue. The discussion of the comparison problem begins in this chapter, where methods are used to investigate whether there appear to be differences across the sub- populations on the numeric variable of interest. In later chapters, inferential methods—con- fidence intervals and hypothesis tests—are used to see whether the differences in samples from the subpopulations can be generalized to the subpopulations as a whole.

Breaking Down by Category

There is arguably no more powerful data analysis technique than breaking down a numeric variable by a categorical variable. The methods in this chapter, espe- cially side-by-side box plots and pivot tables, get you started with this general comparison problem. They allow you to see quickly, with charts and/or numeric summary measures, how two or more categories compare. More advanced tech- niques for comparing across categories are discussed in later chapters.

Fundamental Insight

Stacked and Unstacked Formats There are two possible data formats you might see, stacked and unstacked. Consider salary data on males and females. (There could be other variables in the data set, but they aren’t relevant here.) The data are stacked if there are two “long” variables, Gender and Salary, as indicated in Figure 3.4. The idea is that the male salaries are stacked in with the female salaries. This is the format you will see in the majority of data sets. However, you will occasionally see data in unstacked format, as shown in Figure 3.5. (Note that both

The stacked format is by far the most common. There are one or more long numeric variables and another long variable that specifies which category each observation is in.

Figure 3.4 Stacked Data

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22

A B Gender Salary Male 81600 Female 61600 Female 64300 Female 71900 Male 76300 Female 68200 Male 60900 Female 78600 Female 81700 Male 60200 Female 69200 Male 59000 Male 68600 Male 51900 Female 64100 Male 67600 Female 81100 Female 77000 Female 58800 Female 87800 Male 78900

09953_ch03_ptg01_084-131.indd 90 04/03/19 12:10 PM

3-3 relationships among Categorical Variables and a Numeric Variable 9 1

tables list exactly the same data. See the file Stacked Unstacked Data.xlsx.) Now there are two “short” variables, Female Salary and Male Salary. In addition, it is very possible that the two variables have different lengths. This is the case here because there are more females than males.

Keep this distinction in mind. Some statistical software packages require that the data for a comparison problem be in one of these formats, usually the stacked format. In any case, it is fairly easy to rearrange unstacked data into stacked format or vice versa.

We now return to the baseball data, which is in stacked format, to see which, if any, of the categorical variables makes a difference in player salaries.

Figure 3.5 Unstacked Data

1 2 3 4 5 6 7 8 9

10 11 12 13

A B Female Salary Male Salary

61600 81600 64300 76300 71900 60900 68200 60200 78600 59000 81700 68600 69200 51900 64100 67600 81100 78900 77000 58800 87800

EXAMPLE

3.2 BASEBALL SALARIES The file Baseball Salaries Extra.xlsx contains the same 2018 baseball data examined in the previous chapter. In addition, several extra categorical variables are included:

• Pitcher (Yes for all pitchers, No for the others)

• League (American or National)

• Division (National West, American East, and so on)

• Yankees (Yes if team is New York Yankees, No otherwise)

• Playoff 2017 Team (Yes for the eight teams that made it to the 2017 playoffs, No for the others) • World Series 2017 Team (Yes for Houston Astros and Los Angeles Dodgers, No for others)

Do pitchers (or any other positions) earn more than others? Does one league pay more than the other, or do any divisions pay more than others? How does the notoriously high Yankees payroll compare to the others? Do the successful teams from 2017 tend to have larger 2018 payrolls?

Objective To learn methods for breaking down baseball salaries by various categorical variables.

Solution It is useful to look first at some numeric summary measures for salary. These are the same summary measures from the previous chapter, but now we want to break them down by position or by one of the other categorical variables. You could do this with Excel formulas, but they are not quite as straightforward as before. If you want to create counts or averages,

09953_ch03_ptg01_084-131.indd 91 04/03/19 12:10 PM

you can use Excel’s COUNTIF and AVERAGEIF functions. However, there are no MEDIANIF, MINIF, MAXIF, STDEVIF functions or others you might need. Still, there is a way, using array formulas.

The results appear in Figure 3.6. This table lists selected summary measures for each of the positions in the data set. A typical formula, the standard deviation for catchers, is shown at the bottom. It uses the STDEV.S function but with an IF function inside. This IF function is essentially a filter on catchers, so the function takes the standard deviation of catcher salaries only. However, this is an array formula, so after you type it, you must press Ctrl + Shift + Enter (all three keys at once). This puts curly brackets around the formula, but you do not actually type these curly brackets. If you want to see salaries broken down by team or any other categorical variable, you can easily run this analysis again with a different cate- gorical variable.

Figure 3.6 Summary Measures of Salary by Position

1 2 3 4 5 6 7 8 9

A B C D E F G H I J

10 11 12 13 14 15 16 17 18

Mean Median Min Max Quartile 1 Quartile 3 1st percentile 5th percentile 95th percentile 99th percentile Std Dev Mean Abs Dev

$3,511,360 $1,250,000

$545,000 $22,177,778

$557,471 $3,975,000

$545,000 $546,240

$14,675,000 $22,049,778

$5,009,483 $3,430,724

Position Catcher $12,460,183 $13,262,214

$570,000 $18,666,667 $11,318,607 $17,062,500

$1,091,500 $3,177,500

$18,500,000 $18,633,334

$6,572,010 $4,512,040

Designated Hitter $8,737,151 $5,333,333

$545,000 $30,000,000

$585,000 $14,333,333

$545,336 $546,880

$24,200,000 $28,560,000

$8,977,109 $7,670,583

First Baseman $3,829,218 $1,100,000

$545,000 $34,000,000

$555,000 $5,000,000

$545,000 $545,000

$16,000,000 $27,700,000

$5,650,088 $3,880,380

Pitcher $4,928,349 $2,400,000

$546,900 $24,000,000

$657,000 $8,500,000

$547,008 $547,960

$16,108,674 $20,880,000

$5,489,065 $4,327,677

$3,570,414 $1,025,000

$545,000 $20,000,000

$564,125 $5,112,500

$545,000 $546,305

$15,286,666 $18,120,000

$4,837,949 $3,617,415

$6,029,821 $2,916,666

$545,000 $23,000,000

$565,000 $12,000,000

$546,100 $548,300

$19,319,795 $21,504,000

$6,896,281 $5,900,264

Second Baseman Third BasemanShortstop

Summary stats for Salary broken down by categories of Position

Typical formula Cell C14 =STDEV.S(IF(’Salaries 2018’!$C$2:$C$878=C3,’Salaries 2018’!$J$2:$J$878))

$5,130,183 $1,675,000

$545,000 $34,083,333

$561,500 $6,450,000

$545,000 $546,275

$21,375,000 $27,674,880

$6,978,709 $5,217,029

Outfielder

There are a lot of numbers to digest in Figure 3.6, so it is difficult to get a clear picture of dif- ferences across positions. It is more enlightening to see a graphical summary of this information. There are several types of graphs you can use. Our favorite way, discussed shortly, is to create side-by-side box plots. A second possibility is to use pivot tables and corresponding pivot charts, as discussed later in this chapter.

For now, we illustrate side-by-side box plots. Assuming you have Excel 2016 or later, you can create box plots of Salary by Position. First, select the two columns of data, Position and Salary. Then select a box plot from the Statistical charts list on the Insert ribbon. Excel automatically uses the distinct categories in the Position variable to create a separate box plot for each position. You can interpret each of these box plots as explained in the previous chapter. More importantly, you can compare them.

The results appear in Figure 3.7. Now the differences between positions emerge fairly clearly. A few of the conclusions that can be made follow.

Side-by-side box plots are our favorite graphical way of comparing the distribu- tion of a numeric variable across categories of some categorical variable.

Figure 3.7 Box Plots of Salary by Position

Box Plots of Salary by Position

$35,000,000

$40,000,000

$30,000,000

$25,000,000

$20,000,000

$15,000,000

$10,000,000

$5,000,000

Ou tfi

eld er

Pit ch

er Fir

Ba se

ma n

De sig

na te

Hi tte

rTh ird

Ba se

ma n

Ca tch

Sh or

tst op

Se co

Ba se

ma n

9 2 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 92 04/03/19 12:10 PM

• The salaries for all positions except designated hitter are skewed to the right (mean greater than median, long lines and out- liers to the right).

• As a whole, first basemen tend to be the highest paid players, followed by third basemen. The designated hitters also make a lot, but there are only a few of them in the data set.

• As a whole, pitchers don’t make as much as first basemen and third basemen, but there are a lot of pitchers who are high-earning outliers.

• Except for a few notable exceptions, catchers receive the lowest salaries.

Because these side-by-side box plots are so easy to obtain, you can generate a lot of them to provide insights into the sal- ary data. Several interesting examples appear in Figures 3.8–3.10. These box plots lead to the following conclusions:

• Pitchers make somewhat less than other players, although there are many outliers in each group.

• The Yankees payroll is indeed much larger than the payrolls for the rest of the teams. In fact, it is so large that its stars’ salaries aren’t even considered outliers relative to the rest of the team.

• Aside from the many outliers, the playoff teams from 2017 tend to have slightly larger payrolls than the non-playoff teams. The one question we cannot answer, however, at least not without additional data, is whether these larger payrolls are a cause or an effect of being successful.

You can often create a cate- gorical variable on the fly with an IF formula and then use it for side-by-side box plots. We did this with the Yankees, for example.

Figure 3.8 Box Plots of Salary by Pitcher/Non-Pitcher

Box Plots of Salary by Pitcher/Non-Pitcher

$35,000,000

$40,000,000

$30,000,000

$25,000,000

$20,000,000

$15,000,000

$10,000,000

$5,000,000

$0 No Yes

Figure 3.9 Box Plots of Salary by Yankees/Non-Yankees

Box Plots of Salary by Yankees/Non-Yankees

$35,000,000

$40,000,000

$30,000,000

$25,000,000

$20,000,000

$15,000,000

$10,000,000

$5,000,000

$0 No Yes

3-3 relationships among Categorical Variables and a Numeric Variable 9 3

09953_ch03_ptg01_084-131.indd 93 04/03/19 12:10 PM

Figure 3.10 Box Plots of Salary by Playoff Team/Non- Playoff Team

Box Plots of Salary by Playoff Team/Non-Playoff Team

$35,000,000

$40,000,000

$30,000,000

$25,000,000

$20,000,000

$15,000,000

$10,000,000

$5,000,000

$0 No Yes

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 7. Recall that the file Baseball Salaries Extra.xlsx con-

tains data on 877 Major League Baseball (MLB) players during the 2018 season. Find the mean, median, standard deviation, and first and third quartiles of Salary, broken down by each of the following categories. Comment on your findings. a. Team b. Division c. Whether they played for the Yankees d. Whether they were in the playoffs

8. The file P02_07.xlsx includes data on 204 employees at the (fictional) company Beta Technologies. Find the mean, median, and standard deviation of Annual Salary, broken down by each of the following categories. Com- ment on your findings. a. Gender b. A recoded version of Education, with new values 1 for

Education less than 4, 2 for Education equal to 4, and 3 for Education greater than 4

c. A recoded version of Age, with people aged less than 34 listed as Young, those aged at least 34 and less than 50 listed as Middle-aged, and those aged at least 50 listed as Older

9. The file Golf Stats.xlsx contains data on the 200 top golf- ers each year from 2003 to 2017. (This data set is used in an example in the next section.) For the 2017 data, create a recoded Age variable, with values “Twenties,” “Thirties,” and “Forties,” based on their ages in the 2017 sheet. Then calculate the mean, median, and standard

deviation of the following 2017 variables, broken down by the recoded Age. Comment on whether it appears that golfers peak in their thirties. a. Earnings b. Yards/Drive and Driving Accuracy c. Greens in Regulation d. Putting Average (Golfers want this to be small.)

10. (Requires Excel version 2016 or later for box plots) Recall from Chapter 2 that HyTex Company is a direct marketer of technical products and that the file Catalog Marketing.xlsx contains recent data on 1000 HyTex customers. Find the mean, median, and stan- dard deviation of Amount Spent, broken down by the following variables. Then create side-by-side box plots of Amount Spent, broken down by the same variables. Comment on how the box plots complement the sum- mary measures. a. Age b. Gender c. Close d. Region e. Year of first purchase. (Hint: For this one, use Excel’s

YEAR function to create a Year column.) f. The combination of Married and Own Home. (For

this one, create a code variable, with values from 1 to 4, for the four combinations of Married and Own Home. Alternatively, create a text variable with values such as “Not married, Owns home.”)

11. The file P02_35.xlsx contains data from a survey of 500 randomly selected households. a. Create a new column Has Second Income with values

“Yes” and “No” depending on whether the household has a reported second income.

b. Find the mean, median, and standard deviation of First Income, broken down by the variable you created in

9 4 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 94 04/03/19 12:10 PM

3-3 relationships among Categorical Variables and a Numeric Variable 9 5

part a. Is there any indication that first income tends to be any larger or smaller, or has more or less varia- tion, depending on whether there is a second income?

c. Repeat part b for each of the Monthly Payment and Debt variables.

12. (Requires Excel version 2016 or later for box plots) The file P02_02.xlsx contains data about 256 movies that grossed at least $1 million in 2017. a. Recode Genre so that all genres with fewer than 10

movies are listed as Other. b. Find the mean, median, and standard deviation of

Total US Gross, broken down by the recoded Genre variable. Also, create side-by-side box plots of Total US Gross, again broken down by the recoded Genre variable. Comment on what the results say about the popularity of different genres.

13. (Requires Excel version 2016 or later for box plots) The Wall Street Journal CEO Compensation Study analyzed chief executive officer (CEO) pay from many U.S. com- panies with fiscal year 2008 revenue of at least $5 billion that filed their proxy statements between October 2008 and March 2009. The data are in the file P02_30.xlsx. a. Create a new variable Total 2008, the sum of Salary

2008 and Bonus 2008. (Actually, this is not “total” compensation because it omits the very lucrative com- pensation from stock options.) Also, recode Company Type so that the Technology and Telecommunications are collapsed into a Tech/Telecomm category.

b. Find the mean, median, and standard deviation of Total 2008, broken down by the recoded Company Type. Also, create side-by-side box plots of Total 2008, again broken down by the recoded Company Type. What do the results tell you about differences in level or variability across company types?

14. (Requires Excel version 2016 or later for box plots) The file P02_55.xlsx contains monthly sales (in millions of dollars) of beer, wine, and liquor. The data have not been seasonally adjusted, so there might be seasonal patterns that can be discovered. a. Create a new Month Name variable with values Jan,

Feb, and so on. (Use Excel’s MONTH function and then a lookup table.)

b. Create side-by-side box plots of Total Sales, broken down by Month Name. Is there any evidence of differ- ences across months for either the level of sales or the variability of sales?

15. (Requires Excel version 2016 or later for box plots) The file P03_15.xlsx contains monthly data on the various components of the Consumer Price Index (CPI). The source claims that these data have not been seasonally adjusted. The following parts ask you to check this claim. a. Create a new Month Name variable with values Jan,

Feb, and so on. (Use Excel’s MONTH function and then a lookup table.)

b. Create side-by-side box plots of each component of the CPI (including the All Items variable), broken

down by the Month Name variable from part a. What results would you expect for “not seasonally adjusted” data? Are your results in line with this?

16. (Requires Excel version 2016 or later for box plots) The file P02_11.xlsx contains data on 148 houses that were recently sold in a (fictional) suburban community. The data set includes the selling price of each house, along with its appraised value, square footage, number of bed- rooms, and number of bathrooms. a. Create two new variables, Ratio 1 and Ratio 2. Ratio 1

is the ratio of Appraised Value to Selling Price, and Ratio 2 is the ratio of Selling Price to Square Feet. Identify any obvious outliers in these two Ratio variables.

b. Find the mean, median, and standard deviation of each Ratio variable, broken down by Bedrooms. Also, cre- ate side-by-side box plots of each Ratio variable, again broken down by Bedrooms. Comment on the results.

c. Repeat part b with Bedrooms replaced by Bathrooms. d. If you repeat parts b and c with any obvious outlier(s)

from part a removed, do the conclusions change in any substantial way?

Level B 17. The file P02_32.xlsx contains blood pressures for 1000

people, along with variables that can be related to blood pressure. These other variables have a number of missing values, probably because some people didn’t want to report certain information. For each of the Alcohol, Exercise, and Smoke variables, use StatTools to find the mean, median, and standard deviation of Blood Pressure, broken down by whether the data for that variable are missing. For exam- ple, there should be one set of statistics for people who reported their alcohol consumption and another for those who didn’t report it. Based on your results, does it appear that there is any difference in blood pressure between those who reported and those who didn’t?

18. The file P03_18.xlsx contains the times in the Chicago marathon for the top runners each year (the top 10,000 in 2006 and the top 20,000 in 2007 and 2008). a. Merge the top 1,000 values (that is, the lowest 1,000

times) from each year into a new Combined sheet, and in the new sheet, create a variable Year that lists the year.

b. The Time variable, shown as something like 2:16:12, is really stored as a time, the fraction of day start- ing from midnight. So 2:16:12, for example, which means 2 hours, 16 minutes, and 12 seconds, is stored as 0.0946, meaning that 2:16:12 AM is really 9.46% of the way from midnight to the next midnight. This isn’t very useful. Do whatever it takes to recode the times into a new Minutes variable with two decimals, so that 2:16:12 becomes 136.20 minutes. (Hint: Look up Time functions in Excel’s online help.)

c. Create a new variable Nationality to recode Country as “KEN, ETH,” “USA,” or “Other,” depending on

09953_ch03_ptg01_084-131.indd 95 04/03/19 12:10 PM

9 6 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

whether the runner is from Kenya/Ethiopia (the usual winners), the USA, or some other country.

d. Find the mean, median, standard deviation, and first and third quartiles of Minutes, broken down by Nationality. Also, create side-by-side box plots of Minutes, again broken down by Nationality. Comment on the results.

e. Repeat part d, replacing Nationality by Gender. 19. (Requires Excel version 2016 or later for box plots) The

file P03_19.xlsx contains daily values of the S&P Index from 1970 to mid-2015. It also contains percentage changes in the index from each day to the next. a. Create a new variable President that lists the U.S.

presidents Nixon through Obama on each date. You can look up the presidents and dates online.

b. Find the mean, median, standard deviation, and first and third quartiles of % Change, broken down by President. Also, create side-by-side box plots of % Change, again broken down by President. Comment on the results.

20. The file P02_56.xlsx contains monthly values of indexes that measure the amount of energy necessary to heat or cool buildings due to outside temperatures. (See the explanation in the Source sheet of the file.) These are reported for each

state in the United States and also for several regions, as listed in the Locations sheet, from 1931 to 2000. a. For each of the Heating Degree Days and Cooling

Degree Days sheets, create a new Season variable with values “Winter,” “Spring,” “Summer,” and “Fall.” Winter consists of December, January, and February; Spring consists of March, April, and May; Summer consists of June, July, and August; and Fall consists of September, October, and November.

b. Find the mean, median, and standard deviation of Heating Degree Days (HDD), broken down by Season, for the 48 contiguous states location (code 5999). (Ignore the first and last rows for the given loca- tion, the ones that contain -9999, the code for missing values.) Also, create side-by-side box plots of HDD, broken down by season. Comment on the results. Do they go in the direction you would expect? Do the same for Cooling Degree Days (which has no missing data).

c. Repeat part b for California (code 0499). d. Repeat part b for the New England group of states

(code 5801).

3-4 Relationships Among Numeric Variables This section discusses methods for finding relationships among numeric variables. For example, you might want to examine the relationship between heights and weights of peo- ple, or between salary and years of experience of employees. To study such relationships, we introduce two new summary measures, correlation and covariance, and a type of chart called a scatterplot.

Note that these measures can be applied to any variables that are displayed numeri- cally. However, they are appropriate only for truly numeric variables, not for categorical variables that have been coded numerically. In particular, many people create dummy (0–1) variables for categorical variables such as Gender and then include these dummies in a table of correlations. This is certainly possible, and it is not really “wrong.” However, if you want to investigate relationships involving categorical variables, it is better to employ the tools in the previous two sections.

3-4a Scatterplots We first discuss scatterplots, a graphical method for detecting relationships between two numeric variables.1 Then we will discuss the numeric summary measures, correlation and covariance, in the next subsection. (We do it in this order because correlation and covari- ance make more sense once you understand scatterplots.) A scatterplot is a scatter of points, where each point denotes the values of an observation for two selected variables. The two variables are often labeled generically as X and Y, so a scatterplot is sometimes called an X-Y chart. The whole purpose of a scatterplot is to make a relationship (or the lack of it) apparent. Do the points tend to rise upward from left to right? Do they tend to fall downward from left to right? Does the pattern tend to be linear, nonlinear, or no particular shape? Do any points fall outside the general pattern? The answers to these questions provide information about the possible relationship between the two variables. The process is illustrated in Example 3.3.

In general, don’t use correlations that involve coded categorical variables such as 021 dummies. The methods from the previous two sections are more appropriate.

1 Various terms are used for this type of graph. We use the term scatterplot. You might also see the terms scatter plot (two words), scatter chart, scatter diagram, or scattergram. They all mean exactly the same thing.

09953_ch03_ptg01_084-131.indd 96 04/03/19 12:10 PM

3-4 relationships among Numeric Variables 9 7

EXAMPLE

3.3 GOLF STATS ON THE PGA TOUR For over a decade, the Professional Golf Association (PGA) has kept statistics on all PGA Tour players, and these stats are published on the Web. We imported yearly data from 2003–2017 into the file Golf Stats.xlsx. The file includes an observation for each of the top 200 earners for each year, including age, earnings, events played, rounds played, 36-hole cuts made (only the top scorers on Thursday and Friday get to play on the weekend; the others don’t make the cut), top 10s, and wins. It also includes stats about efficiency in the various parts of the game (driving, putting, greens in regulation, and sand saves), as well as good holes (eagles and birdies) and bad holes (bogies). A sample of the data for 2017 appears in Figure 3.11, with the data sorted in decreasing order of earnings and a few variables not shown. What relationships can be uncovered in these data for any particular year?

Figure 3.11 Golf Stats

4 5 6 7 8 9

Rank Cuts Made Top 10s 12 12

8 7

11 10

7 7 8 7

Greens in Regula on 67.2 70.0 69.5 69.0 68.6 67.0 67.0 63.5 66.4 63.8

Put t ing Average 1.694 1.711 1.755 1.739 1.761 1.721 1.759 1.721 1.785 1.738

Sand Save Pct 48.2 55.8 44.3 50.9 59.1 68.7 53.0 48.2 52.1 58.7

Driving Accuracy 55.1 60.0 57.0 58.6 58.7 63.9 57.9 55.8 67.8 62.8

Yards/Drive 309.7 295.6 315.0 303.3 305.8 300.3 298.6 311.1 289.5 289.9

Earnings $9,921,560 $9,433,033 $8,732,193 $8,380,570 $6,123,248 $6,083,198 $5,866,391 $5,612,397 $4,766,936 $4,396,470

Wins 5 3 4 3 1 1 2 1 1 1

Rounds 79 78 70 76 83 77 90 80 98

100

Events 25 23 20 22 23 21 25 24 28 30

Age 24 24 33 26 23 29 34 27 34 31

Player Just in Thomas Jordan Spieth Dust in Johnson Hideki Matsuyama Jon Rahm Rickie Fowler Marc Leishman Brooks Koepka Kevin Kisner Brian Harman11

10 9 8

1 1 2 3

4 5 6 7

A B C D E F G H I J K L M N

18 19 17 19 21 18 22 19 24 21

Objective To use scatterplots to search for relationships in the golf data.

Solution This example is typical in that there are many numeric variables, and it is up to you to search for possible relationships. A good first step is to ask some interesting questions and then try to answer them with scatterplots. For example, do younger players play more events? Are earnings related to age? Which is related most strongly to earnings: driving, putting, or greens in regulation? Do the answers to these questions remain the same from year to year? This example is all about exploring the data, and we will answer only a few of the questions that could be asked. Fortunately, scatterplots are easy to create, so you can do a lot of exploring very quickly.

To create a scatterplot, you highlight any two variables of interest and select a scatter chart of the top left type from the Insert ribbon. At this point, you will probably want to modify the chart by deleting the legend, inserting some titles, and possibly changing some formatting. Also, you might want to swap the roles of the X and Y variables.

Scatterplots are great for initial exploration of the data. If a scatterplot suggests a relationship between two variables, other methods can then be used to examine this relationship in more depth.

Selecting Multiple Ranges Efficiently

How do you select two long variables such as Age and Earnings? Here are the steps that make it easy. (1) Select the Age label in cell B1. (2) Hold down the Shift and Ctrl keys and press the down arrow key. This selects the Age column. (3) Hold down the Ctrl key and select the Earnings label in cell H1. (4) Hold down the Shift and Ctrl keys and press the down arrow key. Now both columns are selected.

Excel Tip

09953_ch03_ptg01_084-131.indd 97 04/03/19 12:10 PM

Several scatterplots appear in Figures 3.12 through 3.15. The scatterplots in Figure 3.12 indicate the possibly surprising results that age is practically unrelated to the number of events played and earnings. Each scatter is basically a shapeless swarm of points, and a shapeless swarm always indicates “no relationship.” The scatterplots in Figure 3.13 confirm what you would expect. Specifically, players who play in more events tend to earn more, although there are a number of exceptions to this pattern. Also, players who make more 36-hole cuts tend to earn more.

Figure 3.12 Scatterplots of Age Versus Events and Earnings

60.000

50.000

40.000

30.000

20.000

10.000

Age vs Events

0.000 0.000 5.000 10.000 15.000 20.000 25.000 30.000 35.000

60.000

50.000

40.000

30.000

20.000

10.000

Age vs Earnings

0.000 0.000 5,000,000.000 10,000,000.000 15,000,000.000

Figure 3.13 Scatterplots of Earnings Versus Events and Cuts Made

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Events

$0 5.000 10.000 15.000 20.000 25.000 30.000 35.000

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Cuts Made

$0 0.0000.000 5.000 10.000 15.000 20.000 25.000 30.000

Data Labels in Scatterplots

Unfortunately, there is no automatic way to enter a label such as “Dustin Johnson” next to a point in a scatterplot. If you click twice on a point (don’t double-click, but slowly click twice), you can select this point. Then if you right-click, you have the option of adding a data label. However, this data label is always the value of the Y variable. In this case, it would be Dustin’s earnings, not his name.

Excel Tip

Golfers will be particularly interested in the scatterplots in Figures 3.14 and 3.15. The scatterplots in Figure 3.14 indicate a slight positive relationship between earnings and driving length (yards per drive) but almost no relationship between earn- ings and driving accuracy (percentage of fairways hit). The scatterplots in Figure 3.15 indicate a slight positive relationship between earnings and greens hit in regulation and a negative relationship between earnings and putting average. Does this mean that better putters earn less? Absolutely not! The putting stat is the average number of putts per hole, so a lower value is better. Therefore, the downward relationship indicated in the chart is expected. In fact, the driving and putting scatterplots tend to confirm the old saying in golf: Drive for show, putt for dough.

9 8 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 98 04/03/19 12:10 PM

You could obviously ask many more questions about the relationships in this golf data set and then attempt to answer them with scatterplots. For example, are the relationships (or lack of them) in the scatterplots consistent through the years? Or would it be better to use Earnings per Round instead of Earnings as the Y variable? Or are there other variables not shown here, such as the percentage of holed putts of less than 10 feet, that are more highly related to Earnings? You now have a powerful tool, scatterplots, for examining relationships, and the tool is easy to implement. We urge you to use it—a lot.

Figure 3.15 Scatterplots of Earnings Versus Greens in Regulation and Putting Average

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Greens in Regulation

$0 58.0 60.0 62.0 64.0 66.0 68.0 72.070.0

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Putting Average

$0 1.650 1.700 1.750 1.800 1.850 1.900

Figure 3.14 Scatterplots of Earnings Versus Driving Length and Driving Accuracy

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Yards/Drive

$0 260.0 270.0 280.0 290.0 300.0 310.0 320.0

$12,000,000

$10,000,000

$8,000,000

$6,000,000

$4,000,000

$2,000,000

Earnings vs Driving Accuracy

$0 0.0 20.0 40.0 60.0 80.0

Trend Lines in Scatterplots Chapters 10 and 11 discuss regression, a method for quantifying relationships between variables. We can provide a brief introduction to regression at this point by discussing the very useful Trendline tool in Excel. Once you have a scatterplot, Excel enables you to superimpose one of several trend lines on the scatterplot. Essentially, a trend line is a line or curve that “fits” the scatter as well as possible. This could indeed be a straight line, or it could be one of several types of curves. (By the way, you can also superimpose a trend line on a time series graph, exactly as described here for scatterplots.)

To illustrate the Trendline option, we created the scatterplot of driving length versus driving accuracy in Figure 3.16. If you are a golfer, you are probably not surprised to see

Excel allows you to superimpose a trend line, linear or curved, on a scatterplot. It is an easy way to quantify the relationship apparent in the scatterplot.

Figure 3.16 Scatterplot with Trendline Superimposed 320.0

310.0

300.0

290.0

280.0

270.0

Yards/Drive vs Driving Accuracy

260.0 0.0 20.0

y = –1.0704x + 358.56

40.0 60.0 80.0

3-4 relationships among Numeric Variables 9 9

09953_ch03_ptg01_084-131.indd 99 04/03/19 12:10 PM

1 0 0 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

that the longest hitters tend to be less accurate. This scatterplot is definitely downward sloping, and it appears to follow a straight line reasonably well.

Therefore, it is reasonable to fit a linear trend line to this scatterplot, as you see in the figure. To do this, right-click any point on the chart, select Add Trendline, and fill out the resulting dialog box as shown in Figure 3.17. Note that we have checked the Display Equation on Chart option. The equation you see is a regression equation. It states that driving length (y) is 358.56 minus 1.0704 times driving accuracy (x). This line is certainly not a perfect fit because there are many points well above the line and others below the line. Still, it quantifies the downward relationship reasonably well.

Figure 3.17 Trendline Options Dialog Box

The tools in this subsection, scatterplots and trend lines superimposed on scatterplots, are among the most valuable tools you will learn in the book. When you are interested in a possible relationship between two numeric variables, these are the tools you should use first.

09953_ch03_ptg01_084-131.indd 100 04/03/19 12:10 PM

3-4 relationships among Numeric Variables 1 0 1

3-4b Correlation and Covariance Many numeric summary measures for a single variable were discussed in Chapter 2. The two measures discussed in this section, correlation and covariance, involve two variables. Specifically, each measures the strength and direction of a linear relationship between two numeric variables. Intuitively, the relationship is “strong” if the points in a scatterplot cluster tightly around some straight line. If this straight line rises from left to right, the relationship is positive and the measures will be positive numbers. If it falls from left to right, the relationship is negative and the measures will be negative numbers.

To measure the covariance or correlation between two numerical variables X and Y — indeed, to form a scatterplot of X versus Y —X and Y must be “paired” variables. That is, they must have the same number of observations, and the X and Y values for any observation should be naturally paired. For example, each observation could be the height and weight for a particular person, the time in a store and the amount purchased for a particular customer, and so on.

With this in mind, let Xi and Yi be the paired values for observation i, and let n be the number of observations. Then the covariance between X and Y , denoted by Covar(X, Y ), is given by the following formula.

Formula for Covariance

Covar1X, Y2 5 an i 5 1

1Xi 2 X21Yi 2 Y2 n 2 1

(3.1)

Formula for Correlation

Correl1X, Y2 5 Covar1X, Y2 Stdev1X2 3 Stdev1Y2 (3.2)

You will probably never have to use Equation (3.1) directly—Excel has a built-in COVAR function that does it for you—but the formula does indicate what covariance is all about. It is essentially an average of products of deviations from means. If X and Y vary in the same direction, then when X is above its mean, Y will tend to be above its mean, and when X is below its mean, Y will tend to be below its mean. In either case, the product of deviations will be positive—a positive times a positive or a negative times a negative— so the covariance will be positive. The opposite is true when X and Y vary in opposite directions. Then the covariance will be negative.

Covariance has a serious limitation as a descriptive measure because it is very sensitive to the units in which X and Y are measured. For example, the covariance can be inflated by a factor of 1000 simply by measuring X in dollars rather than thousands of dollars. This limits the usefulness of covariance as a descriptive measure, and we will use it very little in the book.2

In contrast, the correlation, denoted by Correl(X, Y ), remedies this problem. It is a unitless quantity that is unaffected by measurement scales. For example, the correlation is the same regardless of whether the variables are measured in dollars, thousands of dollars, or millions of dollars. The correlation is defined by Equation (3.2), where Stdev(X) and Stdev(Y ) denote the standard deviations of X and Y . Again, you will probably never have to use this formula for calculations—Excel does it for you with the built-in CORREL function—but it does show how correlation and covariance are related to one another.

Covariance is too sensitive to the measurement scales of X and Y to make it interpretable, so it is better to rely on correlation, which is unaffected by measure- ment scales.

2 Don’t write off covariance too quickly, however. If you plan to take a finance course in investments, you will see plenty of covariances.

09953_ch03_ptg01_084-131.indd 101 04/03/19 12:10 PM

1 0 2 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

The correlation is not only unaffected by the units of measurement of the two vari- ables, but it is always between 21 and 11. The closer it is to either of these two extremes, the closer the points in a scatterplot are to a straight line, either in the negative or positive direction. On the other hand, if the correlation is close to 0, the scatterplot is typically a “cloud” of points with no apparent relationship. However, although it is not common, it is also possible that the points are close to a curve and have a correlation close to 0. This is because correlation is relevant only for measuring linear relationships.

When there are several numeric variables in a data set, it is useful to create a table of covariances and/or correlations. Each value in the table then corresponds to a particular pair of variables. However, we first make three important points about the roles of scatter- plots, correlations, and covariances.

• A correlation is a single-number summary of a scatterplot. It never conveys as much information as the full scatterplot; it only summarizes the information in the scatterplot. Still, it is often more convenient to report a table of correlations for many variables than to report an unwieldy number of scatterplots.

• You are usually on the lookout for large correlations, those near 21 or 11. But how large is “large”? There is no generally agreed-upon cutoff, but by looking at a number of scatterplots and their corresponding correlations, you will start to get a sense of what a correlation such as 20.5 or 10.7 really means in terms of the strength of the linear relationship between the variables. (In addition, a concrete meaning will be given to the square of a correlation in Chapters 10 and 11.)

• Do not even try to interpret covariances numerically except possibly to check whether they are positive or negative. For interpretive purposes, concentrate on correlations.

Correlation is useful only for measuring the strength of a linear relationship. Strongly related variables can have correlation close to 0 if the relationship is nonlinear.

Scatterplots Versus Correlations

It is important to remember that a correlation is a single-number measure of the linear relationship between two numeric variables. Although a correlation is a very useful measure, it is hard to imagine exactly what a correlation such as 0.3 or 0.8 actually means. In contrast, a scatterplot of two numeric variables indicates the relationship between the two variables very clearly. In short, a scatterplot conveys much more information than the corresponding correlation.

Fundamental Insight

EXAMPLE

3.3 GOLF STATS (CONTINUED) In the previous subsection, you saw how relationships between several of the golf variables can be detected with scatterplots. What further insights are possible by looking at correlations between these variables?

Objective To use correlations to understand relationships in the golf data.

Solution With the many numeric variables in the golf data set, it is indeed unwieldy to create scatterplots for all pairs of variables (unless you have an add-in like StatTools), but it is fairly easy to create a table of correlations. You could use Excel’s CORREL function to calculation each correlation, but if there are many variables, the following steps allow you to fill a table of correla- tions much more quickly. (Refer to Figure 3.18 as you read this.)

1. Highlight all columns of variables to be correlated, including their variable names at the top, and create range names for them with the Create from Selection button on the Formulas ribbon.

2. Copy the variable names to the top row and left column of your correlation table. 3. Create a copyable formula involving the CORREL and INDIRECT functions to calculate all correlations in the table.

09953_ch03_ptg01_084-131.indd 102 04/03/19 12:10 PM

The resulting table of correlations appears in Figure 3.18. The typical formula shown at the bottom was copied to the entire table. The INDIRECT function refers to the range names created in step 1 to get the ranges for the CORREL function. How- ever, note that range names replace illegal characters like / and spaces with underscores. Therefore, the corresponding changes had to be made to the variable names in the table. Also, there are missing data in some of the variables. When calculating the correlation between any two variables, the CORREL function ignores any rows with missing data for these two variables.

Figure 3.18 Correlations for Golf Data

1 2 3 4 5 6 7 8 9

A B C D E F G H I J K L

10 11 12 13 14 15 16

Age Events Rounds Cuts_Made Earnings Yards_Drive Driving_Accuracy Greens_in_Regulation Putting_Average Sand_Save_Pct

Typical formula Cell C5

1.000 –0.085 –0.098 –0.105 –0.191 –0.381

0.341 0.020 0.226

–0.071

–0.085 1.000 0.943 0.698 0.156

–0.036 –0.002

0.035 0.095

–0.128

–0.098 0.943 1.000 0.888 0.334 0.124 0.045 0.241

–0.045 –0.041

–0.105 0.698 0.888 1.000 0.546 0.272 0.058 0.367

–0.179 0.058

–0.191 0.156 0.334 0.546 1.000 0.387

–0.081 0.308

–0.438 0.130

–0.381 –0.036

0.124 0.272 0.387 1.000

–0.590 0.330

–0.044 –0.089

0.341 –0.002

0.045 0.058

–0.081 –0.590

1.000 0.233 0.167

–0.085

0.020 0.035 0.241 0.367 0.308 0.330 0.233 1.000 0.228

–0.160

0.226 0.095

–0.045 –0.179 –0.438 –0.044

0.167 0.228 1.000

–0.267

–0.071 –0.128 –0.041

0.058 0.130

–0.089 –0.085 –0.160 –0.267

1.000

Age Events Rounds Earnings Yards_Drive Greens_in_Regulation Putting_Average Sand_Save_PctDriving_Accuracy

Table of correlations

Cuts_Made

=CORREL(INDIRECT($B5),INDIRECT(C$3))

You can ignore the 1.000 values along the diagonal because a variable is always perfectly correlated with itself. Besides these, you are typically looking for relatively large values, either positive or negative. When the table is fairly large, conditional formatting is useful. For example, you can format all correlations between 0.6 and 0.999 as red and all correlations between 20.999 and 20.5 as green, as has been done here. There are three large positive values, all above 0.69, involving events, rounds, and cuts made. None of these should come as a surprise. There is only one large negative correlation, the one between driving length and driving accuracy, and you already saw the corresponding scatterplot in Figure 3.16. So if you want to know what a correlation of approximately 20.6 really means, you can look at the scatterplot in this figure. It indicates a definite downward trend, but there is still quite a lot of variability around the best-fitting straight line.

Again, a correlation is only a summary of a scatterplot. Therefore, you can learn more about any interesting-looking correlations by creating the corresponding scatterplot. For example, the scatterplot corresponding to the 0.888 correlation between Cuts Made and Rounds appears in Figure 3.19. (We also superimposed a trend line.) This chart shows the strong lin- ear relationship between cuts made and rounds played, but it also shows that there is still considerable variability around the best-fitting straight line, even with a correlation as large as 0.888.

You typically scan a table of correlations for the large correlations, either positive or negative. Conditional formatting is useful, especially if the table is a large one.

Figure 3.19 Scatterplot of Rounds Versus Cuts Made

0 0 5 10 15 20

y = 3.7577x + 18.11

25 30

100

120 Rounds vs Cuts Made

3-4 relationships among Numeric Variables 1 0 3

09953_ch03_ptg01_084-131.indd 103 04/03/19 12:10 PM

1 0 4 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

b. Create a separate table of correlations for each of the selected years that includes Earnings/Event, Yards/Drive, Driving Accuracy, Greens in Regulation, Putting Average, Sand Save Pct, and Birdies/Round. Explain whether these correlations help answer the questions posed above.

c. There is a saying in golf: “Drive for show, putt for dough.” Create a separate set of scatterplots for each of the selected years of Earnings/Event (Y axis) versus each of Yards/ Drive, Driving Accuracy, and Putting Average. Discuss whether these scatterplots tend to support the saying.

25. The file P03_25.xlsx contains data about 211 movies released in 2006 and 2007. The question to be explored in this problem is whether the total gross for a movie can be predicted from how it does in its first week or two. a. Create a table of correlations between the five variables

7-day Gross, 14-day Gross, Total US Gross, Interna- tional Gross, and US DVD Sales. Does it appear that the last three variables are related to either of the first two?

b. Explore the basic question further by creating a scatter- plot of each of Total US Gross, International Gross, and US DVD Sales (Y axis) versus each of 7-day Gross and 14-day Gross (X axis)—six scatterplots in all. Do these support the claim that you can tell how well a movie will do by seeing how it does in its first week or two?

26. The file P02_39.xlsx contains SAT and ACT test scores by state for high school graduates of the 2016– 2017 school year. These are broken down by reading/ writing and math/science scores. The file also lists the percentages of students taking these two tests by state. Create correlations and scatterplots to explore the following relationships and comment on the results. a. The relationship between the total SAT score and the

percentage taking the SAT exam. b. The relationship between the ERW and Math compo-

nents of the SAT exam. c. The relationship between the composite ACT score

and the percentage taking the ACT exam. d. The relationship between the four components of the

ACT exam. 27. The file P02_16.xlsx contains traffic data from 256

weekdays on four variables. Each variable lists the number of arrivals during a specific 5-minute period of the day. a. What would it mean, in the context of traffic, for the

data in the four columns to be positively correlated? Based on your observations of traffic, would you expect positive correlations?

b. Create a table of correlations and check whether these data behave as you would expect.

28. The file P02_11.xlsx contains data on 148 houses that were recently sold in a (fictional) suburban community. The data set includes the selling price of each house, along with its appraised value, square footage, number of bedrooms, and number of bathrooms. a. Create a table of correlations between all of the

variables. Comment on the magnitudes of the correla- tions. Specifically, which of the last three variables,

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 21. The file P02_07.xlsx includes data on 204 employees at

the (fictional) company Beta Technologies. a. Create a table of correlations between the variables

Age, Prior Experience, Beta Experience, Education, and Annual Salary. Which of the first four of these variables is most highly correlated (in a positive direction) with Annual Salary?

b. Create scatterplots of Annual Salary (Y axis) versus each of Age, Prior Experience, Beta Experience, and Education.

c. For the variable from part a most highly correlated with Annual Salary, create a (linear) trend line in its scatterplot with the corresponding equation shown in the chart. What does this equation imply about the relationship between the two variables? Be specific.

22. The file P03_22.xlsx lists financial data on movies released from 1980 to 2011 with budgets of at least $20 million. a. Reduce the size of this data set by deleting all movies

with a budget of more than $100 million. Also, delete all movies where US Gross and/or Worldwide Gross is listed as Unknown.

b. For the remaining movies, create a table of correla- tions between the variables Budget, US Gross, and Worldwide Gross. Comment on the results. Are there any surprises?

c. For the movies remaining after part a, create a scat- terplot of Worldwide Gross (Y axis) versus US Gross and another scatterplot of US Gross (Y axis) versus Budget. Briefly explain any patterns you see in these scatterplots. Do they seem to be consistent with the corresponding correlations?

23. The file P02_10.xlsx contains midterm and final exam scores for 96 students in a corporate finance course. a. Do the students’ scores for the two exams tend to go

together, so that those who do poorly on the midterm tend to do poorly on the final, and those who do well on the midterm tend to do well on the final? Create a scat- terplot, along with a correlation, to answer this question.

b. Superimpose a (linear) trend line on the scatterplot, along with the equation of the line. Based on this equation, what would you expect a student with a 75 on the midterm to score on the final exam?

24. Recall that the file Golf Stats.xlsx contains data on the 200 top golfers each year from 2003 to 2017. The question to be explored in this problem is what drives earnings, and whether this is consistent from year to year. a. For at least three of the years, create two new vari-

ables, Birdies/Round and Earnings/Event. The latter is potentially a better measure of earnings because some players enter more events than others.

09953_ch03_ptg01_084-131.indd 104 04/03/19 12:10 PM

3-4 relationships among Numeric Variables 1 0 5

Square Feet, Bedrooms, and Bathrooms, are highly correlated with Selling Price?

b. Create four scatterplots to show how the other four variables are related to Selling Price. In each, Selling Price should be on the Y axis. Are these in line with the correlations in part a?

c. You might think of the difference, Selling Price minus Appraised Value, as the “error” in the appraised value, in the sense that this difference is how much more or less the house sold for than the appraiser expected. Find the correlation between this difference and Selling Price, and find the correlation between the absolute value of this difference and Selling Price. If either of these cor- relations is reasonably large, what is it telling us?

Level B 29. The file P03_29.xlsx contains daily prices of four pre-

cious metals: gold, silver, platinum, and palladium. The question to be explored here is whether changes in these commodities move together through time. a. Create time series graphs of the four series. Do the

series appear to move together? b. Create four new difference variables, one for each

metal. Each should list this month’s price minus the previous month’s price. Then create time series graphs of the differences.

c. Create a table of correlations between the differences created in part b. Based on this table, comment on whether the changes in the prices of these metals tend to move together over time.

d. For all correlations in part c above 0.6, create the cor- responding scatterplots of the differences (for exam- ple, gold differences versus silver differences). Do these, along with the time series graphs from parts a and b, provide a clearer picture of how these series move together over time?

30. The file P03_30.xlsx contains daily data on exchange rates of various currencies versus the U.S. dollar. It is of interest to financial analysts and economists to see whether exchange rates move together through time. You could find the correlations between the exchange rates themselves, but it is often more useful with time series data to check for correlations between differences from day to day. a. Create a column of differences for each currency. b. Create a table of correlations between all of the origi-

nal variables. Then on the same sheet, create a second table of correlations between the difference variables. On this same sheet, enter two cutoff values, one pos- itive such as 0.6 and one negative such as 20.5, and use conditional formatting to color all correlations (in both tables) above the positive cutoff green and all correlations below the negative cutoff red. Do it so that the 1’s on the diagonal are not colored.

c. Based on the second table and your coloring, can you conclude that these currencies tend to move together in the same direction? If not, what can you conclude?

31. The file P02_35.xlsx contains data from a survey of 500 randomly selected (fictional) households. a. Create a table of correlations between the last five

variables (First Income to Debt). On the sheet with these correlations, enter a “cutoff ” correlation such as 0.5 in a blank cell. Then use conditional format- ting to color green all correlations in the table at least as large as this cutoff, but don’t color the 1’s on the diagonal. The coloring should change auto- matically as you change the cutoff. This is always a good idea for highlighting the “large” correlations in a correlations table.

b. Do some investigation to see how missing values are handled when calculating correlations. There are two basic possibilities (and both of these are options in some software packages). First, it could delete all rows that have missing values for any variables and then calculate all of the correlations based on the remaining data. Second, when it creates the correla- tion for any pair of variables, it could (temporarily) delete only the rows that have missing data for these two variables and then calculate the correlation on what remains for these two variables. Why would you prefer the second option?

32. We have indicated that if you have two categorical vari- ables and you want to check whether they are related, the best method is to create a crosstabs, possibly with the counts expressed as percentages. But suppose both categorical variables have only two categories and these variables are coded as dummy 0–1 variables. Then there is nothing to prevent you from finding the correlation between them with the same Equation (3.2) from this section, that is, with Excel’s CORREL function. How- ever, if we let C1i, j2 be the count of observations where the first variable has value i and the second variable has value j, there are only four joint counts that can have any bearing on the relationship between the two vari- ables: C10,02, C10,12, C11,02, and C11,12. Let C1112 be the count of 1s for the first variable and let C2112 be the count of 1s for the second variable. Then it is clear that C1112 5 C11,02 1 C11,12 and C2112 5 C10,12 1 C11,12, so C1112 and C2112 are determined by the joint counts. It can be shown algebraically that the correlation between the two 091 variables is

nC11,12 2 C1112C2112 !C11121n 2 C11122 !C21121n 2 C21122

To illustrate this, the file P03_32.xlsx contains two 091 variables. (The values were generated randomly.) Create a crosstabs to find the required counts, and use the above formula to calculate the correlation. Then use StatTools (or Excel’s CORREL function) to find the correlation in the usual way. Do your two results match? (Again, we do not necessarily recommend finding correlations between 091 variables. A cross- tabs is more meaningful and easier to interpret.)

09953_ch03_ptg01_084-131.indd 105 04/03/19 12:10 PM

1 0 6 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

3-5 Pivot Tables We now discuss one of Excel’s most powerful—and easy-to-use—tools, pivot tables. Pivot tables allow you to break the data down by categories so that you can see, for example, average sales by gender, by region of country, by time of day, or any combination of these. Sometimes pivot tables are used to display counts, such as the number of customers broken down by gender and region of country. These tables of counts, often called crosstabs or contingency tables, have been used by statisticians for years. However, crosstabs typically list only counts, whereas pivot tables can list counts, sums, averages, and other summary measures.

It is easiest to understand pivot tables by means of examples, so we illustrate several possibilities in the following example.

EXAMPLE

3.4 CUSTOMER ORDERS AT ELECMART The file Elecmart Sales.xlsx (see Figure 3.20) contains data on 400 customer orders during a period of several months for the fictional Elecmart company.3 There are several categorical variables and several numeric variables. The categorical variables include the day of week, time of day, region of country, type of credit card used, gender of customer, and buy category of the customer (high, medium, or low) based on previous behavior. The numeric variables include the number of items ordered, the total cost of the order, and the price of the highest-priced item purchased. How can the manager of Elecmart use pivot tables to summarize the data so that she can understand the buying patterns of her customers?

Objective To use pivot tables to break down the customer order data by a number of categorical variables.

Figure 3.20 Elecmart Data

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15

A B C D E F G H I J Date Day Time Region Card Type Gender Buy Category Items Ordered Total Cost High Item

Sun Sun Sun Sun Sun Sun Mon Mon Mon Tue Tue Tue Wed Wed

Morning Morning

Evening Evening Evening Morning Morning

A�ernoon

A�ernoon A�ernoon A�ernoon A�ernoon

Morning

West West West Northeast West Northeast West South West Midwest Northeast South Northeast Northeast

ElecMart Other ElecMart Other ElecMart Other Other Other Other Other ElecMart Other ElecMart Other

Female Female Female Female Male Female Male Male Male Female Female Male Male Male

High Medium Medium Low Medium Medium Low High Low Low Medium Medium High Low

4 1 5 1 4 5 1 4 2 1 2 2 3 1

$136.97 $25.55

$113.95 $6.82

$147.32 $142.15

$18.65 $178.34

$25.83 $18.13 $54.52 $61.93

$147.68 $27.24

$79.97 $25.55 $90.47

$6.82 $83.21 $50.90 $18.65

$161.93 $15.91 $18.13 $54.38 $56.32 $96.64 $27.24

4-Mar 4-Mar 4-Mar 4-Mar 4-Mar 4-Mar 5-Mar 5-Mar 5-Mar 6-Mar 6-Mar 6-Mar 7-Mar 7-Mar

3 Users of previous editions of the book will notice that the dates have been changed. This changes a few values in the reported pivot tables.

09953_ch03_ptg01_084-131.indd 106 04/03/19 12:10 PM

Solution First, we preview the results you can obtain. Pivot tables are useful for breaking down numeric variables by categories, or for counting observations in categories and possibly expressing the counts as percentages. So, for example, you might want to see how the average total cost for females differs from the similar average for males. Or you might simply want to see the per- centage of the 400 sales made by females. Pivot tables allow you to find such averages and percentages easily.

Actually, you could find such averages or percentages without using pivot tables. For example, you could sort on gender and then find the average of the Female rows and the average of the Male rows. However, this takes time, and more complex breakdowns are even more difficult and time-consuming. They are all easy and quick with pivot tables. Besides that, the result- ing tables can be accompanied with corresponding charts that require virtually no extra effort to create. Pivot tables are a man- ager’s dream. Fortunately, Excel makes them a manager’s reality.

We begin by building a pivot table to find the sum of Total Cost broken down by time of day and region of country. Although we show this in a number of screen shots, just to help you get the knack of it, the process takes only a few seconds after you gain some experience with pivot tables.

To start, click the PivotTable button at the far left on the Insert ribbon (see Figure 3.21). This produces the dialog box in Figure 3.22. The top section allows you to specify the table or range that contains the data. (You can also specify an external data source, but we will not cover this option here.) The bot- tom section allows you to select the location where you want the results to be placed. If you start by selecting any cell inside the data set, Excel’s guess for the table or range is usually correct, although you can override it if necessary. Make sure the

Pivot tables are perfect for breaking down data by cate- gories. Many people refer to this as “slicing and dicing” the data.

Beginning in Excel 2013, there is a Recommended Pivot Tables option. It guesses which pivot tables you might want.

Figure 3.21 PivotTable Button on Insert Ribbon

Figure 3.22 Create PivotTable Dialog Box

3-5 pivot tables 1 0 7

09953_ch03_ptg01_084-131.indd 107 04/03/19 12:10 PM

range selected for this example is A1:J401. This selected range should include the variable names at the top of each column. Then click OK. Note that with these settings, the pivot table will be placed in a new worksheet with a generic name such as Sheet1. You will probably want to rename it something like Pivot Table.

This produces a blank pivot table, as shown in Figure 3.23. Also, assuming any cell inside this blank pivot table is selected, the PivotTable Tools “super tab” is visible. This super tab has two ribbons, Analzye and Design. (The Analyze ribbon was named Options in Excel 2007 and 2010.) The Analyze ribbon appears in Figure 3.24, and the Design ribbon appears in Figure 3.25. Each of these has a variety of buttons for manipulating pivot tables, some of which we will explore shortly. Finally, the PivotTable Fields pane in Figure 3.26 is visible. By default, it is docked at the right of the screen, but you can move it if you like.

Figure 3.23 Blank Pivot Table

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21

A B C D

Click in this area to work with the PivotTable report

PivotTable1

Figure 3.24 PivotTable Analyze Ribbon

Figure 3.25 PivotTable Design Ribbon

1 0 8 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 108 04/03/19 12:10 PM

Note that the two pivot table ribbons and the PivotTable Fields pane are visible only when the active cell is inside a pivot table. If you click outside the pivot table, say, in cell D1, all three of these will disappear. Don’t worry. You can get them back by selecting any cell inside the pivot table.

The PivotTable Fields pane indicates that a pivot table has four areas. These are for Filters, Rows, Columns, and Values. They correspond to the four areas in Figure 3.26 where you can put fields.4

A Rows field has categories that go down the left side of a pivot table, a Columns field has categories that go across the top of a pivot table, a Filters field lets you filter the pivot table by its categories, and a Values field contains the data you want to summarize. Typically (but not always), you will place categorical variables in the Filters, Rows, and/or Columns areas, and you will place numeric variables in the Values area.

In the present example, check the Time, Region, and Total Cost boxes in the upper half of the PivotTable Fields pane. You immediately get the pivot table in Figure 3.27. It shows the sum of Total Cost, broken down by time of day and region of coun- try. For example, the total cost of orders in the morning in the South was $3835.86, and the total cost of orders in the morning (over all regions) was $18,427.31.

Excel applies two rules to variables checked at the top of the PivotTable Fields pane:

• When you check a text variable or a date variable in the field list, it is added to the Rows area. • When you check a numeric variable in the field list, it is added to the Values area and summarized with the Sum function.

This is exactly what happens when you check Time, Region, and Total Cost. However, this is just the beginning. With very little work, you can do a lot more. Some of the many possi- bilities are explained in the remainder of this example.

First, notice that the pivot table in Figure 3.27 has both Rows fields, Time and Region, in column A. This is one of three possible layouts: Compact, Outline, or Tabular. These are available from the Report Layout dropdown list on the Design ribbon. When you create a

Figure 3.26 PivotTable Fields Pane

There are three different layouts for pivot tables, but the differences are relatively minor. Ultimately, it is a matter of taste.

4 In discussing pivot tables, Microsoft uses the term field rather than variable, so we will do so as well.

3-5 pivot tables 1 0 9

09953_ch03_ptg01_084-131.indd 109 04/03/19 12:10 PM

1 1 0 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

pivot table (in an .xlsx file), you get the compact layout by default. If you would rather have the tabular or outline layout, it is easy to switch to them. In particular, the tabular layout, shown in Figure 3.28, is closer to what was used in pre-2007 versions of Excel. (Outline layout, not shown here, is very similar to tabular layout except for the placement of its subtotals.)

A 1 2 3 Row Labels Sum of Total Cost

24265.6A�ernoon

18834.3Evening

18427.31Morning

Midwest 3187.16 Northeast South

8159.78 5729.72

West 7188.94

Midwest 2552.89 Northeast South

5941.49 3864.12

West 6475.8

Midwest 3878.22 Northeast South

5084.57 3835.86

West

4 5 6 7 8 9

10 11 12 13 14 15 16 17

5628.6618 19 61527.21Grand Total

Figure 3.27 Sum of Total Cost by Time and Region (Compact Layout)

A 1 2 3

3187.16A�ernoon

A�ernoon Total

Evening Total

2552.89Evening

18834.3 Morning

Morning Total

Midwest 8159.78Northeast

South 5729.72 7188.94West

Midwest Northeast South West

24265.6

5941.49 3864.12

6475.8

3878.22 5084.57 3835.86

4 5 6 7 8 9

10 11 12 13 14 15 16 17

5628.66 18 19

18427.31 Grand Total

Time Sum of Total CostRegion

61527.21

Figure 3.28 Sum of Total Cost by Time and Region (Tabular Layout)

One significant advantage to using tabular (or outline) layout instead of compact layout is that you can see which fields are in the Rows and Columns areas. Take another look at the pivot table in Figure 3.27. It is fairly obvious that categories such as Afternoon and Morning have to do with time of day and that categories such as Midwest and South have to do with region of country. However, there are no labels that explicitly name the Rows fields. In contrast, the tabular layout in Figure 3.28 names them explicitly, and it makes filtering more transparent. Some critics have been very vocal about their dislike for compact layout due to the lack of meaningful labels, and we tend to agree. Therefore, the remaining screenshots in this chapter show tabular layout.

09953_ch03_ptg01_084-131.indd 110 04/03/19 12:10 PM

1 2 3 4 5 6 7 8

A B C D FE

Grand Total

Morning A�ernoon Evening

Sum of Total Cost Time Midwest

Region Northeast South West Grand Total

3878.22 3187.16 2552.89 9618.27

5084.57 8159.78 5941.49

19185.84

3835.86 5729.72 3864.12 13429.7

5628.66 7188.94

6475.8 19293.4

18427.31 24265.6 18834.3

61527.21

Figure 3.29 Placing Region in the Columns Area

1 2 3 4 5 6 7 8 9 10 11 12 13 14

A B C D FE G

Morning

Grand Total

Morning Total

Female Male

Sum of Total Cost Time Gender Midwest

Region Northeast South West Grand Total

A�ernoon

A�ernoon Total Evening

Evening Total

2880.13 998.09

3878.22 2092.39 1094.77 3187.16 1411.32 1141.57 2552.89 9618.27

2864.79 2219.78 5084.57

6206.8 1952.98 8159.78 1103.44 4838.05 5941.49

19185.84

2774.01 1061.85 3835.86 5073.03

656.69 5729.72 1181.66 2682.46 3864.12 13429.7

4211.19 1417.47 5628.66 5210.59 1978.35 7188.94 1885.91 4589.89

6475.8 19293.4

12730.12 5697.19

18427.31 18582.81

5682.79 24265.6 5582.33

13251.97 18834.3

61527.21

Figure 3.30 Adding a Second Field to the Rows Area

Note that times of day in column A are now in chronological order, not in alphabetical order as they were by default. It is easy to make this change. Select the top of the Morning cell (A7) until the cursor becomes a four-way arrow and drag it above Afternoon.

You can also categorize by a third field such as Gender. As before, if you check Gender in the PivotTable Fields pane, it goes to the Rows area by default, but you could then drag it to another area. The pivot table in Figure 3.30 shows the result of placing Gender in the Rows area. For example, the sum of Total Cost for morning orders by females in the West is 4211.19. You can place as many fields as you like in any area. The only downside is that the pivot table starts to get cluttered. If you want to remove a field from the pivot table, just uncheck its its item in the top part of the PivotTable Fields pane or drag it off from the bottom part of the pane.

If you checked the Gender field to get Figure 3.30, you'll notice that Gender is below Time in the Rows box. This makes Time the "outer" Rows field and Gender the "inner" Rows field in the pivot table. To reverse their roles, drag Gender above Time in the Rows box. This leads to the pivot table in Figure 3.31. The two pivot tables provide slightly different views of the same summarization, and you can choose the one you like best. This ability to categorize by multiple fields and rearrange the fields as you like is a big reason why pivot tables are so powerful and useful—and easy to use.

Changing Field Settings You can change various settings in the Field Settings dialog box. You can get to this dialog box in at least two ways. First, there is a Field Setting button on the Analyze ribbon. Second, you can right-click any of the pivot table cells and select Field Settings. The field settings are particularly useful for fields in the Values area.

For now, right-click any number in the pivot table in Figure 3.31 and select Value Field Settings to obtain the dialog box in Figure 3.32. This allows you to choose how you want to summarize the Total Cost variable—by Sum, Average, Count,

Changing the locations of fields in pivot tables is easy. We favor dragging the fields to the various areas, but you can experiment with the other options.

Changing Locations of Fields Starting with the pivot table in Figure 3.28, you can choose where to place either Time or Region; it does not have to be in the Rows area. To place the Region variable in the Columns area, for example, drag the Region button from the Rows area of the PivotTable Fields pane to the Columns area. The pivot table changes automatically, as shown in Figure 3.29. (This is called pivoting.)

3-5 pivot tables 1 1 1

09953_ch03_ptg01_084-131.indd 111 04/03/19 12:10 PM

or several others. You can also click the Number Format button to choose from the usual number formatting options, and you can click the Show Values As tab to display the data in various ways (more on this later). If you choose Average and format as currency with two decimals, the resulting pivot table appears as in Figure 3.33. Now each number is the aver- age of Total Cost for all orders in its combination of categories. For example, the average of Total Cost for all morning orders by females in the South is $146.00, and the average of all orders by femailes in the South is $143.31.

Figure 3.31 Pivot Table with Rows Fields Reversed 1

2 3 4 5 6 7 8 9 10 11 12 13

A B C D FE G

Female Total

Grand Total

Morning A�ernoon Evening

9618.27 19185.84 13429.7 19293.4 61527.21

Sum of Total Cost Gender Time Midwest

Region Northeast South West Grand Total

Female

Male Total

Male

2880.13 2092.39 1411.32 6383.84

998.09 1094.77 1141.57 3234.43

2864.79 6206.8

1103.44 10175.03

2219.78 1952.98 4838.05 9010.81

2774.01 5073.03 1181.66

9028.7 1061.85

656.69 2682.46

4401

4211.19 5210.59 1885.91

11307.69 1417.47 1978.35 4589.89 7985.71

12730.12 18582.81

5582.33 36895.26

5697.19 5682.79

13251.97 24631.95

The key to summarizing the data the way you want it summarized is the Value Field Settings dialog box. Get used to it because you will use it often.

Figure 3.32 Value Field Settings Dialog Box

Figure 3.33 Pivot Table with Average of Total Cost 1

2 3 4 5 6 7 8 9 10 11 12 13

A B C D FE G

Female

Female Total

Grand Total

Morning A�ernoon Evening

Average of Total C Gender Time Midwest Northeast South West Grand Total

Male

Male Total

$160.01 $123.08 $176.42 $148.46 $124.76 $121.64 $103.78 $115.52 $135.47

$143.24 $172.41 $183.91 $164.11 $170.75 $162.75 $172.79 $170.02 $166.83

$146.00 $153.73 $107.42 $143.31 $132.73 $109.45 $167.65 $146.70 $144.41

$150.40 $192.98 $171.45 $171.33 $141.75 $141.31 $148.06 $145.19 $159.45

$149.77 $164.45 $155.06 $157.67 $146.08 $138.60 $154.09 $148.39 $153.82

Region

1 1 2 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 112 04/03/19 12:10 PM

Filtering When you place a categorical field in the Rows, Columns, or Filters area, all its categories show by default. But it is often use- ful to filter out, or hide, some of these categories. This lets you focus on the categories of most interest. There are several ways to filter; you can choose the method you prefer.

1. Click the dropdown arrow next to any Rows or Columns field in the pivot table and check the categories you want to show. Alternatively, choose Label Filters from the dropdown list, which provide several useful options.

2. Add a field to the Filters area. Then filter with its dropdown arrow. 3. Add a slicer (or a timeline for date fields) from the Filters group on the Analyze ribbon. These are graphical items for

making filters more transparent. (Slicers were introduced in Excel 2010. Timelines were introduced in Excel 2013.)

Figures 3.34, 3.35, and 3.36 illustrate these three options. Figure 3.34 uses filters on Time and Region to hide the After- noon, South, and West categories. Note that the grand totals are only for the filtered data. For example, $142.91 in cell B7 is the average of Total Cost only for the morning and evening orders in the Midwest. Also note the filter signs on the Time and Region dropdown arrows. You can click either of these to change or remove the filters.

1 2 3 4 5 6 7

A B C D

Grand Total

Morning Evening

Average of Total Cost Region Northeast Grand Total

$149.16 $134.36 $142.91

$154.08 $174.75 $164.57

$151.91 $160.27 $155.87

Time Midwest

Figure 3.34 Filtering on the Fields in the Rows and Columns Areas

Figure 3.35 uses a filter for Day in the Filters area. It indicates that multiple categories were checked in the Day dropdown list. However, without opening this list, you can’t tell which days were selected. (They are Friday and Saturday.) This lack of transparency is a drawback of filters in the Filters area.

Figure 3.35 Using a Day Filter in the Filters Area 1

2 3 4 5 6 7 8

A B C D FE

Grand Total

Morning Afternoon Evening

Day (Multiple Items)

Average of Total Cost Region Northeast South West Grand Total

$175.20 $133.13 $144.21 $149.93

$172.40 $200.93 $161.78 $182.85

$151.51 $167.23 $138.82 $153.05

$162.39 $182.74 $133.56 $159.96

$166.50 $173.95 $144.15 $163.52

Time Midwest

Figure 3.36 illustrates a slicer on Day and a timeline on Date. When you select Insert Slicer from the Analyze ribbon, you can check as many fields as you like. You get a slicer for each field you check. (For a timeline, you are likely to select only a single date variable.) To choose categories from a slicer, select as many as you want in the usual way: hold down the Ctrl key to select multiple items or the Shift key to select contiguous items. It is clear from this slicer and this timeline that only orders on Fridays and Saturdays during March and April are summarized. This is the advantage of slicers and timelines. They make filters transparent. By the way, to create Figure 3.36, starting from Figure 3.35, we unchecked the Day field in the PivotTable Fields pane. However, this did not remove the filter on Day. When we inserted the slicer on Day, Friday and Saturday were already selected. Just keep this in mind. When you delete a filtered field from a pivot table, the filter is still remembered.

Slicers and timelines aren’t really part of a pivot table in the sense that when they are selected, the PivotTable Fields pane isn’t opened. However, they each have their own ribbon. For example, when you select a slicer, a Slicer Tools Options tab appears, with an associated ribbon. This lets you make several cosmetic changes to the slicer.

Sorting You can sort by categories or values in a pivot table. We already mentioned one sort, where Morning was moved above Afternoon by dragging. However, dragging was required only because the “natural” order Morning, Afternoon, Evening is not

3-5 pivot tables 1 1 3

09953_ch03_ptg01_084-131.indd 113 04/03/19 12:10 PM

alphabetical or reverse alphabetical order. More commonly, you will sort by clicking the dropdown arrow next to a Rows or Columns field and using one of the Sort options at the top. These include A-Z, Z-A, and More Sort Options. You can experi- ment with the latter.

Sorting by values— the values in the body of the pivot table—is also possible. For example, if you select cell E5 in Figure 3.36 and click the A-Z button on the Data ribbon, not only will cells E5, E6, and E7 will be sorted in increasing order, but (as you would hope) the entire range A5:F7 will be rearranged as well. This is just one possibility. You can also right-click a value in the pivot table, select Sort and then More Sort Options to get the dialog box in Figure 3.37. This lets you sort in increasing or decreasing order from top to bottom or from left to right. Don’t be afraid to experiment with these options. You can always press Ctrl+z to undo your sort.

IHG

Day

Sun

Mon

Tue

Wed

Thu

Fri

Sat

FEDCBA

Date

Mar - Apr 2018 MONTHS

2018

JAN FEB MAR APR MAY JUN JUL

Morning

A�ernoon

Evening

Grand Total

$167.57

$93.57

$218.29

$131.62

$113.10

$101.05

$94.31

$105.13

$161.47

$143.65

$148.04

$147.72

$119.90

$86.22

$149.87

$134.26

$135.61

$111.34

$139.91

$127.45

Midwest

Region

Northeast South West Grand Total

Average of Total Cost

Time

Figure 3.36 Filtering with a Slicer and a Timeline

Figure 3.37 More Sort Options for Values

Pivot Charts It is easy to accompany pivot tables with pivot charts. These charts are really Excel charts, but they are better because they adapt automatically to the underlying pivot table. If you make a change to the pivot table, such as pivoting the Rows and Columns fields, the pivot chart makes the same change automatically. To create a pivot chart, click anywhere inside the pivot table, select the PivotChart button on the Analyze ribbon, and select a chart type. That’s all there is to it. The resulting pivot

1 1 4 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

09953_ch03_ptg01_084-131.indd 114 04/03/19 12:10 PM

chart (using the default column chart option) for the pivot table in Figure 3.36 appears in Figure 3.38. If you decide to pivot the Rows and Columns fields, the pivot chart changes automatically, as shown in Figure 3.39. Note that the categories on the horizontal axis are always based on the Rows field, and the categories in the legend are always based on the Columns field.

$0.00 Morning A�ernoon Evening

$250.00

$200.00

$150.00

$100.00

$50.00

Time

Region

Midwest

Northeast

South

West

Average of Total CostFigure 3.38 Pivot Chart Based on Pivot Table

$0.00 Midwest Northeast South West

$250.00

$200.00

$150.00

$100.00

$50.00

Region

Time

Morning

A�ernoon

Evening

Average of Total CostFigure 3.39 Pivot Chart after Pivoting Rows and Columns Fields

Note that when you activate a pivot chart, the PivotTable Tools “super tab” changes to PivotChart Tools. This super tab includes three ribbons for manipulating pivot charts: Analyze, Design, and Format. There is not enough space here to discuss the many options on these ribbons, but they are intuitive and easy to use. As usual, don’t be afraid to experiment.

Multiple Variables in the Values Area Multiple variables can be placed in the Values area. In addition, a given variable in the Values area can be summarized by more than one summarizing function. This can create a rather busy pivot table, so we indicate our favorite way of doing it. Starting with the pivot table in Figure 3.40, drag Total Cost in the top of the PivotTable Fields pane (the item that is already checked) to the Values area. The bottom part of the PivotTable Fields pane should now appear as in Figure 3.41, and the pivot table should now appear as in Figure 3.42. Note in particular the Values button in the Rows area of Figure 3.41. This button controls the placement of the data in the pivot table. You have a number of options for this button: (1) leave it where it is, (2) drag it above the Time button, (3) drag it to the Columns area, below the Region button, or (4) drag it to the Columns area, above the Region button. You can experiment with these options, but we tend to prefer option (2), which leads to the pivot table in Figure 3.43.

3-5 pivot tables 1 1 5

09953_ch03_ptg01_084-131.indd 115 04/03/19 12:10 PM

1 1 6 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

In a similar manner, you can experiment with the buttons in the Values area. However, the effect here is less striking. If you drag the Sum of Total Cost button above the Average of Total Cost button in the field list, the effect is simply to switch the ordering of these summaries in the pivot table.

Summarizing by Count The field in the Values area, whatever it is, can be summarized by the Count function. This is useful when you want to know, for example, how many of the orders were placed by females in the South. When summarizing by Count, the key is to under- stand that the field placed in the Values area is irrelevant, so long as you summarize it by the Count function. To illustrate, start

Morning Average of Total Cost $149.16 3878.22

3187.16

2552.89

$122.58

$134.36

$154.08 5084.57

8159.78

5941.49

$170.00

$174.75

$142.07 3835.86

5729.72

3864.12

$146.92

$143.12

$148.12 5628.66

7188.94

6475.8

$175.34

$154.19

$148.61 18427.31

24265.6

18834.3

$157.57

$154.38

$135.47 9618.27

$166.83 $144.41 $159.45 19185.84 13429.7 19293.4 61527.21

$153.82

Average of Total Cost

Sum of Total Cost

Afternoon

Evening

Total Sum of Total Cost Total Average of Total Cost

Time Values Region Midwest Northeast South West Grand Total

1 2 3 4 5 6 7 8 9

10 11 12

A B C ED F G

Figure 3.42 Pivot Table with Two Values Fields

1 2 3 4 5 6 7 8

A B C D FE

Grand Total

Morning A�ernoon Evening

Average of Total Cost Region Northeast South West Grand Total

$149.16 $122.58 $134.36 $135.47

$154.08 $170.00 $174.75 $166.83

$142.07 $146.92 $143.12 $144.41

$148.12 $175.34 $154.19 $159.45

$148.61 $157.57 $154.38 $153.82

Time Midwest

Figure 3.40 Basic Pivot Table

Filters Columns

Region

Values

Rows

Time Average of To...

Sum of Total ...

Drag fields between areas below:Figure 3.41 Choosing Two Values Fields

09953_ch03_ptg01_084-131.indd 116 04/03/19 12:10 PM

3-5 pivot tables 1 1 7

with the pivot table in Figure 3.40, where Total Cost is summarized with the Average function. Next, right-click any number in the pivot table, select Value Field Settings, and select the Count from the Summarize Values By options. The default Custom Name you will see in this dialog box, Count of Total Cost, is misleading, because Total Cost has nothing to do with the counts obtained. Therefore, we like to change this Custom Name label to Count. While in the Value Field Settings dialog box, change the number format to Number with zero decimals. The resulting pivot table appears in Figure 3.44. For example, 27 of the 400 orders were placed in the morning in the South, and 115 of the 400 orders were placed in the Northeast. (Do you now see why the counts have nothing to do with Total Cost?) This type of pivot table, with counts for various categories, is the same as the crosstabs discussed in Section 3-2. However, these counts can be created much more quickly with a pivot table.

When data are summarized by counts, there are a number of ways they can be dis- played. The pivot table in Figure 3.44 shows “raw counts.” Depending on the type of information you want, it might be more useful to display the counts as percentages. Three particular options are typically chosen: as percentages of total, as percentages of row totals, and as percentages of column totals. When shown as percentages of total, the percentages in the table sum to 100%; when shown as percentages of row totals, the percentages in each row sum to 100%; when shown as percentages of column totals, the percentages in each column sum to 100%. Each of these options can be useful, depending on the question you are trying to answer. For example, if you want to know whether the daily pattern of orders varies from region to region, showing the counts as percentages of column totals is useful so that you can compare columns. But if you want to see whether the regional ordering pattern varies by time of day, showing the counts as percentages of row totals is useful so that you can compare rows.

Figure 3.43 Rearranged Pivot Table with Two Values Fields

Morning $149.16

3878.22 3187.16 2552.89

$122.58 $134.36

$154.08

5084.57 8159.78 5941.49

$170.00 $174.75

$142.07

3835.86 5729.72 3864.12

$146.92 $143.12

$148.12

5628.66 7188.94

6475.8

$175.34 $154.19

$148.61

18427.31 24265.6 18834.3

$157.57 $154.38

$135.47 9618.27

$166.83 $144.41 $159.45 19185.84 13429.7 19293.4 61527.21

$153.82

Afternoon Evening Morning Afternoon Evening

Total Sum of Total Cost

Sum of Total Cost

Average of Total Cost

Total Average of Total Cost

TimeValues Region Midwest Northeast South West Grand Total

1 2 3 4 5 6 7 8 9

10 11 12

A B C ED F G

Counts can be displayed in a number of ways. You should choose the way that best answers the question you are asking.

Morning Afternoon Evening Grand Total

Count Time

Region Midwest Northeast South West Grand Total

1 2 3 4 5 6 7 8

26 26 19 71

33 48 34

115

27 39 27 93

38 41 42

121

124 154 122 400

A B C ED FFigure 3.44 Pivot Table with Counts

To display the counts as percentages, right-click any number in the pivot table, select Show Values As, and select the option you want. (You can also get to these options from the Show Values As tab in the Value Field Settings dialog box.) For example, if you choose % of Column Totals, the resulting pivot table and corresponding pivot chart appear in Figure 3.45. As you can see by comparing columns, the pattern of regional orders varies somewhat by time of day.

09953_ch03_ptg01_084-131.indd 117 04/03/19 12:10 PM

1 1 8 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

Sometimes it is useful to see the raw counts and the percentages. This can be done easily by dragging any variable to the Values area, summarizing it by Count, and displaying it as “Normal.” Figure 3.46 shows one possibility, where we have changed the custom names of the two Count variables to make them more meaningful. Alternatively, the counts and percentages could be shown in two separate pivot tables.

Figure 3.45 Pivot Table and Pivot Chart with Counts As Percentages of Column Totals

A B C ED F

Morning Afternoon Evening

50.00%

40.00%

30.00%

20.00%

10.00%

0.00%

Region

Time

Count

Midwest

Northeast

South

West

Morning Afternoon Evening

Count Time

Region Midwest Northeast South West Grand Total

Grand Total

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20

36.62% 36.62% 26.76%

100.00%

29.03% 41.94% 29.03%

100.00%

31.40% 33.88% 34.71%

100.00%

31.00% 38.50% 30.50%

100.00%

28.70% 41.74% 29.57%

100.00%

Right-Clicking to Choose Options

We keep saying to make changes in the Value Field Settings dialog box. However, you can also make changes directly by right-clicking a value in the pivot table. For example, when you right-click a number in the Values area, you see Number Format, Summarize Values By, and Show Values As menu items, among others.

Pivot Table Tip

Grouping Finally, categories in a Rows or Columns variable can be grouped. This is especially useful when a Rows or Columns variable has many distinct values. Because a pivot table creates a row or column for each distinct value, the results can be unwieldy. We present two possibilities. First, suppose you want to summarize Sum of Total Cost by Date. If you have Excel 2016 or later, the dates will automatically be grouped in some way, probably by month or by quarter. (Prior to Excel 2016, they weren’t grouped at all. You saw a row for each separate date.) In any case, you can right-click any date and select Group. Then you can choose the grouping you prefer.

For example, starting with a blank pivot table, we dragged Date to the Rows area and Total Cost to the Values area to get the pivot table in Figure 3.47 (shown in compact layout). By right-clicking any of the month labels and selecting Group, the dialog box in Figure 3.48 opens. The dates are currently grouped by Month and Day, meaning that if you expand any month in column A, you will see a row for each day in that month. If you select Quarter, Month, and Day in Figure 3.48, you will see the pivot table in Figure 3.49 (after collapsing the months).

09953_ch03_ptg01_084-131.indd 118 04/03/19 12:10 PM

3-5 pivot tables 1 1 9

Figure 3.46 Pivot Table with Raw Counts and Percentages of Column Totals

MorningCount

% of column

71 100.00% 100.00% 100.00% 100.00% 100.00%

115 93 121 400

Afternoon Evening Morning Afternoon Evening

Total Count Total % of column

TimeValues Region Midwest Northeast South West Grand Total

1 2 3 4 5 6 7 8 9

10 11 12

26 26 19

40.32% 33.14% 26.54%

33 48 34

26.50% 42.53% 30.97%

27 39 27

28.56% 42.66% 28.77%

38 41 42

29.17% 37.26% 33.56%

124 154 122

29.95% 39.44% 30.61%

A B C ED F G

1 2 3 4 5 6 7 8

A B

Grand Total

Mar Apr May Jun

$102.97 $146.79 $174.89 $191.51 $153.82

Row Labels Average of Total Cost

Figure 3.47 Default Grouping on Date

Figure 3.48 Grouping Dialog Box

09953_ch03_ptg01_084-131.indd 119 04/03/19 12:10 PM

1 2 0 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

As a second possibility for grouping, suppose you want to see how the average of Total Cost varies by the amount of the highest priced item in the order. You can drag Total Cost to the Values area, summarized by Average, and drag High Item to the Rows area. Because High Item has nearly 400 distinct values, the resulting pivot table is virtually worthless. Therefore, group- ing is useful. This time there are no natural groupings as there are for a date variable, so it is up to you to create the groupings. Excel provides the suggested grouping in Figure 3.50, but you can override it. For example, changing the bottom entry to 50 leads to the pivot table in Figure 3.51. Some experimentation is typically required to obtain the grouping that presents the results in the most appropriate way.

Figure 3.49 Another Grouping on Date 1

2 3 4 5 6 7 8 9

10 11 12

A B

Mar

Apr May Jun

Qtr1

Qtr2 Qtr1 Total

Qtr2 Total

Row Labels Average of Total Cost

Grand Total

$102.97 $102.97

$146.79 $174.89 $191.51 $170.54 $153.82

Grouping on Dates

Suppose you have multiple years of data and you would like a monthly grouping such as January 2017 through December 2019. If you simply select Months in the Grouping dialog box, all of the Januaries, for example, will be lumped together. The trick is to select both Months and Years in the dialog box.

Pivot Table Tip

A 1 2 3

$72.78 $139.66 $172.71 $253.55 $324.26 $328.92 $361.53 $415.17

4 6.82-56.82 56.82-106.82 106.82-156.82 156.82-206.82 206.82-256.82 256.82-306.82 306.82-356.82 356.82-406.82

5 6 7 8 9

10 11 12

High Item Average of Total Cost

Grand Total $153.82

Figure 3.51 Pivot Table after Grouping by 50 on High Item

Figure 3.50 Grouping Dialog Box for a Non-Date Variable

09953_ch03_ptg01_084-131.indd 120 04/03/19 12:10 PM

By now, we have illustrated the pivot table features that are most commonly used. Be aware, however, that there are many more features available. These include, but are not limited to, the following:

• Showing/hiding subtotals and grand totals (check the Layout options on the Design ribbon)

• Dealing with blank rows, that is, categories with no data (right-click any number, choose PivotTable Options, and check the options on the Layout & Format tab)

• Displaying the data behind a given number in a pivot table (double-click any number in the Values area to get a new worksheet)

• Formatting a pivot table with various styles (check the style options on the Design ribbon)

• Moving or renaming pivot tables (check the PivotTable and Action groups on the Analyze ribbon)

• Refreshing pivot tables as the underlying data changes (check the Refresh dropdown list on the Analyze ribbon)

• Creating pivot table formulas for calculated fields or calculated items (check the Formulas dropdown list on the Analyze ribbon)

• Basing pivot tables on external databases (see the next chapter)

Not only are these (and other) features available, but Excel usually provides more than one way to implement them. The suggestions given here are just some of the ways they can be implemented. The key to learning pivot table features is to exper- iment. There are entire books written about pivot tables, but we don’t recommend them. You can learn a lot more, and a lot more quickly, by experimenting with data such as the Elecmart data. Don’t be afraid to mess up. Pivot tables are very forgiv- ing, and you can always start over.

How Excel Stores Pivot Table Data

When you create a pivot table, Excel stores a snapshot of your source data in memory in a pivot cache. The amount of memory depends on the size of the data source, but it can be large. Fortunately, if you create another pivot table based on the same data source, Excel is intelligent enough to use the same pivot cache, thus conserving memory. (This sharing behavior began with Excel 2010. There is a way to mimic the pre-2010 behavior, that is, to create a separate pivot cache for each pivot table, but it is not discussed here.)

Excel Tip

We complete this section by providing one last example to illustrate how pivot tables can answer business questions quickly.

EXAMPLE

3.5 FROZEN LASAGNA DINNERS The file Lasagna Triers.xlsx contains data on over 800 potential customers being tracked by a (fictional) company that has been marketing a frozen lasagna dinner. The file contains a number of demographics on these customers, as indicated in Figure 3.52: their age, weight, income, pay type, car value, credit card debt, gender, whether they live alone, dwelling type, monthly number of trips to the mall, and neighborhood. It also indicates whether they have tried the company’s frozen lasagna. The company wants to understand why some potential customers are triers and others are not. Does gender make a difference? Does income make a difference? In general, what distinguishes triers from nontriers? How can the company use pivot tables to explore these questions?

3-5 pivot tables 1 2 1

09953_ch03_ptg01_084-131.indd 121 04/03/19 12:10 PM

1 2 2 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

Objective To use pivot tables to explore which demographic variables help to distinguish lasa- gna triers from nontriers.

Solution The key is to set up a pivot table that shows counts of triers and nontriers for different categories of any of the potential explanatory variables. For example, one such pivot table shows the percentages of triers and nontriers for males and females separately. If the percentages are different for males than for females, the company will know that gender has an effect. On the other hand, if the percentages for males and females are about the same, the company will know that gender does not make much of a difference.

The typical pivot table should be set up as shown in Figure 3.53. The Rows variable is any demographic variable you want to investigate—in this case, Gender. The Columns variable is Have Tried (Yes or No). The Values variable can be any variable, as long as it is expressed as a count. Finally, it is useful to show these counts as percentage of row totals. This way you can easily look down column C to see whether the percentage in one category (Female) who have tried the product is any different from the percentage in another category (Male) who have tried the product. As you can see, males are somewhat more likely to try the product than females: 60.92% versus 54.27%. This is also apparent from the associated pivot chart.

Pivot tables, with counts in the Values area, are a great way to discover which variables have the largest effect on a Yes/No variable.

Figure 3.52 Lasagna Trier Data

1 2 3 4 5 6 7 8 9

10 11

A B C D E F G H I J K L M Person Age Weight Income Pay Type Car Value CC Debt Gender Live Alone Dwell Type Mall Trips Nbhd Have Tried

1 48 175 65500 Hourly 2190 3510 Male No Home 7 East No 2 33 202 29100 Hourly 2110 740 Female No Condo 4 East Yes 3 51 188 32200 Salaried 5140 910 Male No Condo 1 East No 4 56 244 19000 Hourly 700 1620 Female No Home 3 West No 5 28 218 81400 Salaried 26620 600 Male No Apt 3 West Yes 6 51 173 73000 Salaried 24520 950 Female No Condo 2 East No 7 44 182 66400 Salaried 10130 3500 Female Yes Condo 6 West Yes 8 29 189 46200 Salaried 10250 2860 Male No Condo 5 West Yes 9 28 200 61100 Salaried 17210 3180 Male No Condo 10 West Yes

10 29 209 9800 Salaried 2090 1270 Female Yes Apt 7 East Yes

Figure 3.53 Pivot Table and Pivot Chart for Examining the Effect of Gender

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Yes

Female Male

1 2 3 4 5 6 7 8 9 10 11 12 13

A B C D E F G H I

Female Male Grand Total

45.73%

42.17% 39.08%

54.27%

57.83% 60.92%

100.00%

100.00% 100.00%

Count Gender

Have Tried No Yes Grand Total

Once this generic pivot table and associated pivot chart are set up, you can easily explore other demographic variables by swapping them for Gender. For example, Figure 3.54 indicates that people who live alone are (not surprisingly) much more likely to try this frozen microwave product than people who don’t live alone.

As another example, Figure 3.55 indicates that people with larger incomes are slightly more likely to try the product. There are two things to note about this income pivot table. First, because there are so many individual income values, grouping is useful. You can experiment with the grouping to get the most meaningful results. Second, you should be a bit skeptical about the last group, which has 100% triers. It is possible that there are only one or two people in this group. (It turns out that there are four.) For this reason, it is a good idea to create two pivot tables of the counts, one showing percentage of row totals and one showing the raw counts. This second pivot table is shown at the bottom of Figure 3.55.

09953_ch03_ptg01_084-131.indd 122 04/03/19 12:10 PM

The problem posed in this example is a common one in real business situations. One variable indicates whether people are in one group or another (triers or nontriers), and there are a lot of other variables that could potentially explain why some people are in one group and others are in the other group. There are a number of sophisticated techniques for attacking this classification problem, and some of these are discussed in Chapter 17. However, you can go a long way toward understanding which variables are important by the simple pivot table method illustrated here.

Figure 3.54 Pivot Table and Pivot Chart for Examining the Effect of Live Alone

80.00%

70.00%

60.00%

50.00%

40.00%

30.00%

20.00%

10.00%

0.00% No Yes

Yes

IHGFEDCBA 1 2 3 4 5 6 7 8 9

10 11 12 13

Count Live Alone

Have Tried No Yes Grand Total

100.00% 100.00% 100.00%

45.57% 25.52% 42.17%

No Yes Grand Total

54.43% 74.48% 57.83%

Figure 3.55 Pivot Table and Pivot Chart for Examining the Effect of Income

IHGFEDCBA 1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Count Income

Have Tried No Yes Grand Total

0-49999 50000-99999 100000-149999 150000-199999 Grand Total

Count Income

Have Tried No Yes Grand Total

0-49999 50000-99999 100000-149999 150000-199999 Grand Total

307 152

32 4

495

563 243

46 4

856

256 91 14

361

45.47% 37.45% 30.43%

0.00% 42.17%

54.53% 62.55% 69.57%

100.00% 57.83%

Count

Income

Have Tried

Count

Income

Have Tried

120.00% 100.00%

80.00% 60.00% 40.00% 20.00%

0.00%

0- 49

99 9

50 00

0- 99

99 9

10 00

00 -1

49 99

15 00

00 -1

99 99

Yes

0- 49

99 9

400 300 200 100

50 00

0- 99

99 9

10 00

00 -..

15 00

00 -..

. No

Yes

100.00% 100.00% 100.00% 100.00% 100.00%

3-5 pivot tables 1 2 3

09953_ch03_ptg01_084-131.indd 123 04/03/19 12:10 PM

1 2 4 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

41. The Wall Street Journal CEO Compensation Study analyzed CEO pay from many U.S. companies with fiscal year 2008 revenue of at least $5 billion that filed their proxy statements between October 2008 and March 2009. The data are in the file P02_30.xlsx. a. Create a pivot table and a corresponding pivot chart

that simultaneously shows average of Salary 2008 and average of Bonus 2008, broken down by Company Type. Comment on any striking results in the chart.

b. In the Data sheet, create a new column, Total 2008, which is the sum of Salary 2008 and Bonus 2008. Then create two pivot tables and corresponding pivot charts on a single sheet. The first should show the counts of CEOs broken down by Company Type, and the second should simultaneously show the average of Total 2008, the minimum of Total 2008, and the maximum of Total 2008, all broken down by Company Type. Comment on any striking results in these charts.

42. One pivot table element we didn’t explain is a calcu- lated item. This is usually a new category for some categorical variable that is created from existing cate- gories. It is easiest to learn from an example. Open the file Elecmart Sales.xlsx from this section, create a pivot table, and put Day in the Rows area. Proceed as follows to create two new categories, Weekday and Weekend. a. Select any day and select Calculated Item from the

Formulas dropdown list on the PivotTable Tools Options ribbon. This will open a dialog box. Enter Weekend in the Name box and enter the formula = Sat 1 Sun in the formula box. (You can dou- ble-click the items in the Items list to help build this formula.) When you click OK, you will see Weekend in the pivot table.

b. Do it yourself. Create another calculated item, Week- day, for Mon through Fri.

c. Filter out all of the individual days from the row area, so that only Weekday and Weekend remain, and then find the sum of Total Cost for these two new cate- gories. How can you check whether these sums are what you think they should be? (Notes about calcu- lated items: First, if you have Weekend, Weekday, and some individual days showing in the Rows area, the sum of Total Cost will double-count these individual days, so be careful about this. Second, be aware that if you create a calculated item from some variable such as Day, you are no longer allowed to drag that vari- able to the Filters area.)

43. Building on the previous problem, another pivot table element we didn’t explain is a calculated field. This is usually a new numerical variable built from numerical variables that can be summarized in the Values area. It acts somewhat like a new column in the spreadsheet data, but there is an important difference. Again, it is easiest to learn from an example. Open the file Elecmart Sales.xlsx and follow the instructions below.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 33. Solve problem 1 with pivot tables and create correspond-

ing pivot charts. Express the counts as percentages of row totals. What do these percentages indicate about this particular data set? Then repeat, expressing the counts as percentages of column totals.

34. Solve problem 2 with pivot tables and create corre- sponding pivot charts. Express the counts as percent- ages of row totals. What do these percentages indicate about this particular data set? Then repeat, expressing the counts as percentages of column totals.

35. Solve problem 3 with pivot tables and create correspond- ing pivot charts. Express the counts as percentages of row totals. What do these percentages indicate about this particular data set? Then repeat, expressing the counts as percentages of columns totals.

36. Solve problem 4 with pivot tables and create corre- sponding pivot charts. Express the counts as percentage of row totals. What do these percentages indicate about this particular data set? Then repeat, expressing the counts as percentages of column totals.

37. Solve problem 7 with pivot tables and create correspond- ing pivot charts. However, find only means and standard deviations, not medians or quartiles. (This is one draw- back of pivot tables. Medians, quartiles, and percentiles are not in the list of summary measures.)

38. Solve problem 8 with pivot tables and create corre- sponding pivot charts. However, find only means and standard deviations, not medians. (This is one drawback of pivot tables. Medians are not among their summary measures.)

39. Solve problem 9 with pivot tables and create corre- sponding pivot charts. However, find only means and standard deviations, not medians. (This is one drawback of pivot tables. Medians are not among their summary measures.)

40. The file P03_40.xlsx contains monthly data on the num- ber of vehicles crossing the border from Mexico into four southwestern states. a. Restructure this data set on a new sheet so that there

are three long columns: Month, State, and Crossings. Essentially, you should stack the original columns B through E on top of one another to get the Crossings column, and you should also indicate which state each row corresponds to in the State column. The Month column should have four replicas of the original Month column.

b. Create a pivot table and corresponding pivot table chart based on the restructured data. It should break down the average of Crossings by Year and State. Comment on any patterns you see in the chart.

09953_ch03_ptg01_084-131.indd 124 04/03/19 12:10 PM

3-5 pivot tables 1 2 5

a. Create a new column in the data, CostPerItem, which is Total Cost divided by Items Ordered. Then create a pivot table and find the average of CostPerItem, bro- ken down by Region. Explain exactly how this value was calculated. Would such an average be of much interest to a manager at Elecmart? Why or why not?

b. Select any average in the pivot table and then select Calculated Field from the Formulas dropdown list on the Analyze ribbon. This will open a dialog box. Enter CF_CostPerItem in the name box (we added CF, for calculated field, because we are not allowed to use the CostPerItem name that already exists), enter the for- mula = TotalCost/ItemsOrdered, and click OK. You should now see a new column in the pivot table, Sum of CF_CostPerItem, with different values than in the Average of CostPerItem column. Do some investiga- tion to understand how each “sum” was calculated. From a manager’s point of view, does it make any sense? (Note on calculated fields: When you summa- rize a calculated field, it doesn’t matter whether you express it as sum, average, max, or any other sum- mary measure. It is calculated in exactly the same way in each case.)

44. The file P02_18.xlsx contains daily values of the S&P Index from 1970 to mid-2015. It also contains percent- age changes in the index from each day to the next. Cre- ate a pivot table with average of % Change in the Values area and Date in the Rows area. You will see every single date, with no real averaging taking place. This problem lets you explore how you can group naturally on a date variable. For each part below, explain the result briefly. (Note that if you are using Excel 2016, grouping by date will be done automatically.) a. Group by Month. b. Group by Year. c. Group by Month and Year (select both in the Group

dialog box). Can you make it show the year averages from part b?

d. Group by Quarter. e. Group by Month and Quarter. Can you make it show

the averages from part c? f. Group by Quarter and Year. g. Group by Month, Quarter, and Year.

45. Using the Elecmart Sales.xlsx file from this section, experiment with slicers as follows. a. Create a pivot table that shows the average of Total

Cost, broken down by Region in the Rows area and Time in the Columns area. Then insert two slicers, one for Region and one for Time. Select the West and Northeast buttons on the Region slicer and the Morning and Afternoon buttons on the Time slicer. Explain what happens in the pivot table.

b. Create a pivot table that shows the average of Total Cost, broken down by Region in the Rows area and

Time in the Columns area. Insert a Day slicer and select the Sat and Sun buttons. Explain what aver- ages are now showing in the pivot table. Verify this by deleting the slicer and placing Days in the Filters area, with Sat and Sun selected.

46. We used the Lasagna Triers.xlsx file in this section to show how pivot tables can help explain which variables are related to the buying behavior of customers. Illus- trate how the same information could be obtained with slicers. Specifically, set up the pivot table as in the exam- ple, but use a slicer instead of a Rows variable. Then set it up exactly as in the example, with a Rows variable, but include a slicer for some other variable. Comment on the type of results you obtain with these two versions. Do slicers appear to provide any advantage in this type of problem?

Level B 47. Solve problem 5 with pivot tables and create corre-

sponding pivot charts. If you first find the quartiles of Salary and Amount Spent (by any method), is it possi- ble to create the desired crosstabs by grouping, without recoding these variables?

48. Solve problem 17 with pivot tables. However, find only means and standard deviations, not medians. This is one drawback of pivot tables. Medians are not among their summary measures. Can you think of a way to calculate medians by category?

49. The file P03_22.xlsx lists financial data on movies released from 1980 to 2011 with budgets of at least $20 million. a. Create three new variables, Ratio1, Ratio2, and

Decade. Ratio1 should be US Gross divided by Budget, Ratio2 should be Worldwide Gross divided by Budget, and Decade should list 1980s, 1990s, or 2000s, depending on the year of the release date. If either US Gross or Worldwide Gross is listed as “Unknown,” the corresponding ratio should be blank. (Hint: For Decade, use the YEAR function to fill in a new Year column. Then use a lookup table to populate the Decade column.)

b. Use a pivot table to find counts of movies by various distributors. Then go back to the data and create one more column, Distributor New, which lists the distrib- utor for distributors with at least 30 movies and lists Other for the rest. (Hint: Use a lookup table to popu- late Distributor New, but also use an IF to fill in Other where the distributor is missing.)

c. Create a pivot table and corresponding pivot chart that shows average and standard deviation of Ratio1, broken down by Distributor New, with Decade in the Filters area. Comment on any striking results.

d. Repeat part c for Ratio2.

09953_ch03_ptg01_084-131.indd 125 04/03/19 12:10 PM

1 2 6 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

50. The file P03_50.xlsx lists NBA salaries for five seasons. (Each NBA season straddles two calendar years.) a. Merge all of the data into a single new sheet called All

Data. In this new sheet, add a new column Season that lists the season, such as 2006–2007.

b. Note that many of the players list a position such as C–F or F–C. Presumably, the first means the player is primarily a center but sometimes plays forward, whereas the second means the opposite. Recode these so that only the primary position remains (C in the first case, F in the second). To complicate matters further, the source lists positions differently in 2007–2008 than in other years. It lists PG and SG (point guard and shooting guard) instead of just G, and it lists SF and PF (small forward and power forward) instead of just F. Recode the positions for this season to be consistent with the other seasons (so that there are only three positions: G, F, and C).

c. Note that many players have (p) or (t) in their Con- tract Thru value. The Source sheet explains this. Create two new columns in the All Data sheet, Years Remaining and Option. The Years Remaining column should list the years remaining in the contract. For

example, if the season is 2004–2005 and the contract is through 2006–2007, years remaining should be 2. The Option column should list Player if there is a (p), Team if there is a (t), and blank if neither.

d. Use a pivot table to find the average Salary by Season. Change it to show average Salary by Team. Change it to show average Salary by Season and Team. Change it to show average Salary by Primary Position. Change it to show average Salary by Team and Primary Position, with filters for Season, Contract Years, Years Remaining, and Option. Comment on any striking findings.

51. The file P02_29.xlsx contain monthly percentages of on-time arrivals at several of the largest U.S. airports. a. Explain why the current format of either data set lim-

its the kind of information you can obtain with a pivot table. For example, does it allow you find the average on-time arrival percentage by year for any selected subset of airports, such as the average for O’Hare, Los Angeles International, and La Guardia?

b. Restructure the data appropriately and then use a pivot table to answer the specific question in part a.

3-6 Conclusion Finding relationships among variables is arguably the most important task in data analysis. This chapter has equipped you with some very powerful tools for detecting relationships. As we have discussed, the tools vary depending on whether the variables are categorical or numerical. (Again, refer to the diagram in the Data Analysis Taxonomy.xlsx file.) Tables and charts of counts are useful for relationships among categorical variables. Summary measures broken down by categories and side-by- side box plots are useful for finding relationships between a categorical and a numerical variable. Scatterplots and correlations are useful for finding relationships among numerical variables. Finally, pivot tables are useful for all types of variables.

Summary of Key Terms TERM EXPLANATION EXCEL PAGES EQUATION Crosstabs (or contingency table)

Table of counts of joint categories of two categorical variables

COUNTIFS function or pivot table

Comparison problem Comparing a numeric variable across two or more subpopulations

Stacked or unstacked data formats

Stacked means long columns, one for categories and another for values; unstacked means a separate values column for each category

Scatterplot (or X-Y chart)

Chart for detecting a relationship between two numeric variables; one point for each observation

Scatter from Insert ribbon

trend line Line or curve fit to scatterplot (or time series graph)

Right-click on chart point, select Add Trendline

09953_ch03_ptg01_084-131.indd 126 04/03/19 12:10 PM

3-6 Conclusion 1 2 7

Problems

Conceptual Questions C.1. When you are trying to discover whether there is a

relationship between two categorical variables, why is it useful to transform the counts in a crosstabs to per- centages of row or column totals? Once you do this, how can you tell if the variables are related?

C.2. Suppose you have a crosstabs of two “Yes/No” cate- gorical variables, with the counts shown as percent- ages of row totals. What will these percentages look like if there is absolutely no relationship between the variables? Besides this case, list all possible types of relationships that could occur. (There aren’t many.)

C.3. If you suspect that a company’s advertising expendi- tures in a given month affect its sales in future months, what correlations would you look at to confirm your suspicions? How would you find them?

C.4. Suppose you have customer data on whether they have bought your product in a given time period, along with various demographics on the custom- ers. Explain how you could use pivot tables to see which demographics are the primary drivers of their “yes/no” buying behavior.

C.5. Suppose you have data on student achievement in high school for each of many school districts. In spreadsheet format, the school district is in column A, and various student achievement measures are in columns B, C, and so on. If you find fairly low correlations (magni- tudes from 0 to 0.4, say) between the variables in these achievement columns, what exactly does this mean?

C.6. In the final round of most professional golf tourna- ments, two players play together in a “twosome.” It often appears that good play or bad play is contagious. That is, if one player in the twosome plays well, the other plays well, and if one player plays badly, the other player plays badly. What data would you collect,

and how would you use the data, to check whether this “contagious” property is true?

C.7. Suppose you have a large data set for some sport. Each row might correspond to a particular team (as in the file P03_57.xlsx on football outcomes, for example) or it might even correspond to a given play. Each row contains one or more measures of success as well as many pieces of data that could be drivers of success. How might you find the most important drivers of success if the success measure is categorical (such as Win or Lose)? How might you find the most important drivers of success if the success measure is numerical and basically continuous (such as Points Scored in basketball)?

C.8. If two variables are highly correlated, does this imply that changes in one cause changes in the other? If not, give at least one example from the real world that illus- trates what else could cause a high correlation.

C.9. Suppose there are two commodities A and B with strongly negatively correlated daily returns, such as a stock and gold. Is it possible to find another commod- ity with daily returns that are strongly negatively cor- related with both A and B?

C.10. In checking whether several times series, such as monthly exchange rates of various currencies, move together, why do most analysts look at correlations between their differences rather than correlations between the original series?

Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 52. Suppose you would like to create a separate table of cor-

relations for each category of a categorical variable. The only alternative, at least with the software you have, is to sort on the categorical variable, insert some blank rows between values of different categories, copy the headings

TERM EXPLANATION EXCEL PAGES EQUATION

Covariance Measure of linear relationship between two numeric variables, but affected by units of measurement

COVAR function 101 3.1

Correlation Measure of linear relationship between two numeric variables, always from 21 to 11

CORREL function 102 3.2

pivot table Table for breaking down data by category; can show counts, averages, or other summary measures

PivotTable from Insert ribbon

108

pivot chart Chart corresponding to a pivot table PivotTable Tools Analyze ribbon

117

Slicers, timelines Graphical elements for filtering in pivot tables

PivotTable Tools Analyze ribbon

127

Key Terms (continued)

09953_ch03_ptg01_084-131.indd 127 04/03/19 12:10 PM

1 2 8 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

to each section, and then ask for correlations from each. Do this with the movie data in the file P03_25.xlsx. Specifically, separate the data into three data sets based on Genre: one for Comedy, one for Drama, and one for all the rest. For this problem, you can ignore the third group. For each of Comedy and Drama, create a table of correlations between 7-day Gross, 14-day Gross, Total US Gross, International Gross, and US DVD Sales. Comment on whether the correlation structure is much different for these two popular genres.

53. The file P03_53.xlsx lists campaign contributions, by number of contributors and contribution amount, by state (including Washington DC) for the four leading con- tenders in the 2008 presidential race. Create a scatterplot and corresponding correlation between Dollar Amount (Y axis) and Contributors for each of the four contend- ers. For each scatterplot, superimpose a linear trend line and show the corresponding equation. Interpret each equation and compare them across candidates. Finally, identify the state for any points that aren’t on or very close to the corresponding trend line.

54. The file P03_54.xlsx lists data for 593 movies released in 2011. Obviously, some movies are simply more pop- ular than others, but success in 2011, measured by 2011 gross or 2011 tickets sold, could also be influenced by the release date. To check this, create a new variable, Days Out, which is the number of days the movie was out during 2011. For example, a movie released on 12/15 would have Days Out equal to 17 (which includes the release day). Create two scatterplots and correspond- ing correlations, one of 2011 Gross (Y axis) versus Days Out and one of 2011 Tickets Sold (Y axis) ver- sus Days Out. Describe the behavior you see. Do you think a movie’s success can be predicted very well just by knowing how many days it has been out?

55. The file P03_55.xlsx lists the average salary for each MLB team from 2004 to 2011, along with the number of team wins in each of these years. a. Create a table of correlations between the Wins col-

umns. What do these correlations indicate? Are they higher or lower than you expected?

b. Create a table of correlations between the Salary col- umns. What do these correlations indicate? Are they higher or lower than you expected?

c. For each year, create a scatterplot and the associated correlations between Wins for that year (Y axis) and Salary for that year. Does it appear that teams are buy- ing their way to success?

d. The coloring in the Wins columns indicates the play- off teams. Create a new Yes/No column for each year, indicating whether the team made it to the playoffs that year. Then create a pivot table for each year showing average of Salary for that year, broken down by the Yes/ No column for that year. Do these pivot tables indicate that teams are buying their way into the playoffs?

56. The file P03_56.xlsx lists the average salary for each NBA team from the 2004–2005 season to the

2009–2010 season, along with the number of team wins each of these years. Answer the same questions as in the previous problem for this basketball data.

57. The file P03_57.xlsx lists the average salary for each NFL team from 2002 to 2009, along with the number of team wins each of these years. Answer the same ques- tions as in problem 55 for this football data.

58. The file P03_58.xlsx lists salaries of MLB players in the years 2007 to 2009. Each row corresponds to a partic- ular player. As indicated by blank salaries, some play- ers played in one of these years, some played in two of these years, and the rest played in all three years. a. Create a new Yes/No variable, All 3 Years, that indi-

cates which players played all three years. b. Create two pivot tables and corresponding pivot

charts. The first should show the count of players by position who played all three years. The sec- ond should show the average salary each year, by position, for all players who played all three years. (For each of these, put the All 3 Years variable in the Filters area.) Explain briefly what these two pivot tables indicate.

c. Consider the data set consisting of only the players who played all three years. Using this data set, cre- ate a table of correlations of the three salary variables. What do these correlations indicate about player salaries?

59. The file P03_59.xlsx lists the results of about 20,000 runners in the 2008 New York Marathon. a. For all runners who finished in 3.5 hours or less, create

a pivot table and corresponding pivot chart of average of Time by Gender. (To get a fairer com-parison in the chart, change it so that the vertical axis starts at zero.) For the same runners, and on the same sheet, create another pivot table and pivot chart of counts by Gender. Comment on the results.

b. For all runners who finished in 3.5 hours or less, create a pivot table and corresponding pivot chart of average of Time by Age. Group by Age so that the teens are in one category, those in their twenties are in another category, and so on. For the same runners, and on the same sheet, create another pivot table and pivot chart of counts of these age groups. Comment on the results.

c. For all runners who finished in 3.5 hours or less, create a single pivot table of average of Time and of counts, broken down by Country. Then filter so that only the 10 countries with the 10 lowest average times appear. Finally, sort on average times so that the fastest countries rise to the top. Guess who the top two are! (Hint: Try the Value Filters for the Country variable.) Comment on the results.

60. The file P02_12.xlsx includes data on the 50 top grad- uate programs in the United States, according to a U.S. News & World Report survey. a. Create a table of correlations between all of the

numerical variables. Discuss which variables are highly correlated with which others.

09953_ch03_ptg01_084-131.indd 128 04/03/19 12:10 PM

3-6 Conclusion 1 2 9

b. The Overall score is the score schools agonize about. Create a scatterplot and corresponding correlation of each of the other variables versus Overall, with Over- all always on the Y axis. What do you learn from these scatterplots?

61. Recall from an example in the previous chapter that the file Supermarket Transactions.xlsx contains over 14,000 transactions made by supermarket customers over a period of approximately two years. Set up a sin- gle pivot table and corresponding pivot chart, with some instructions to a user (like the supermarket manager) in a text box, on how the user can get answers to any typical question about the data. For example, one possibility (of many) could be total revenue by product department and month, for any combination of gender, marital status, and homeowner. (The point is to get you to explain pivot table basics to a nontechnical user.)

62. The file P03_15.xlsx contains monthly data on the vari- ous components of the Consumer Price Index. a. Use Excel formulas to create differences for each of

the variables. b. Create a times series graph for each CPI component,

including the All Items component. Then create a time series graph for each difference variable. Comment on any patterns or trends you see.

c. Create a table of correlations between the differences. Comment on any large correlations (or the lack of them).

d. Create a scatterplot for each difference variable versus the difference for All Items (Y axis). Comment on any patterns or outliers you see.

Level B 63. The file P03_63.xlsx contains financial data on 85

U.S. companies in the Computer and Electronic Prod- uct Manufacturing sector (NAICS code 334) with 2009 earnings before taxes of at least $10,000. Each of these companies listed R&D (research and development) expenses on its income statement. Create a table of cor- relations between all of the variables and use conditional formatting to color green all correlations involving R&D that are strongly positive or negative. (Use cutoff values of your choice to define “strongly.”) Then create scatterplots of R&D (Y axis) versus each of the other most highly correlated variables. Comment on any pat- terns you see in these scatterplots, including any obvi- ous outliers, and explain why (or if) it makes sense that these variables are highly correlated with R&D. If there are highly correlated variables with R&D, can you tell which way the causality goes?

64. The file P03_64.xlsx lists monthly data since 1950 on the well-known Dow Jones Industrial Average (DJIA), as well as the less well-known Dow Jones Transporta- tion Average (DJTA) and Dow Jones Utilities Average (DJUA). Each of these is an index based on 20 to 30 leading companies (which change over time).

a. Create monthly differences in three new columns. The Jan-50 values will be blank because there are no Dec-49 values. Then, for example, the Feb-50 differ- ence is the Feb-50 value minus the Jan-50 value.

b. Create a table of correlations of the three difference columns. Does it appear that the three Dow indexes tend to move together through time?

c. It is possible (and has been claimed) that one of the indexes is a “leading indicator” of another. For exam- ple, a change in the DJUA in September might predict a similar change in the DJIA in the following Decem- ber. To check for such behavior, create four “lags” of the difference variables. Do this for each of the three difference variables. You should end up with 12 lag variables. Explain in words what these lag variables contain. For example, what is the Dec-50 lag 3 of the DJIA difference?

d. Create a table of correlations of the three differences and the 12 lags. Use conditional formatting to color green all correlations greater than 0.5 (or any other cutoff you choose). Does it appear that any index is indeed a leading indicator of any other? Explain.

65. The file P03_65.xlsx lists a lot of data for each NBA team for the seasons 2004–2005 to 2008–2009. The variables are divided into groups: (1) Overall success, (2) Offensive, and (3) Defensive. The basic question all basketball fans (and coaches) ponder is what causes suc- cess or failure. a. Explore this question by creating a correlation matrix

with the variable Wins (the measure of success) and all of the variables in groups (2) and (3). Based on these correlations, which five variables appear to be the best predictors of success? (Keep in mind that negative correlations can also be important.)

b. Explore this question in a different way, using the Playoff Team column as a measure of success. Here, it makes sense to proceed as in the Lasagna Triers example in Section 3-5, using the variables in groups (2) and (3) as the predictors. However, these predic- tors are all basically continuous, so grouping would be required for all of them in the pivot table, and grouping is always somewhat arbitrary. Instead, create a copy of the Data sheet. Then for each variable in groups (2) to (13), create a formula that returns 1, 2, 3, or 4, depend- ing on which quarter of that variable the value falls in (1 if it is less than or equal to the first quartile, and so on). (This sounds like a lot of work, but a single copy- able formula will work for the entire range.) Now use these discrete variables as predictors and proceed as in the Lasagna Triers example. List the five variables that appear to be the best (or at least good) predictors of making the playoffs.

66. The file P03_66.xlsx lists a lot of data for each NFL team for the years 2004 to 2009. The variables are divided into groups: (1) Overall success, (2) Team Offense, (3) Pass- ing Offense, (4) Rushing Offense, (5) Turnovers Against,

09953_ch03_ptg01_084-131.indd 129 04/03/19 12:10 PM

1 3 0 C h a p t e r 3 F i n d i n g r e l a t i o n s h i p s a m o n g V a r i a b l e s

(6) Punt Returns, (7) Kick Returns, (8) Field Goals, (9) Punts, (10) Team Defense, (11) Passing Defense, (12) Rushing Defense, and (13) Turnovers Caused. The basic question all football fans (and coaches) ponder is what causes success or failure. Answer the same ques- tions as in the previous problem for this football data, but use all of the variables in groups (2) to (13) as possi- ble predictors.

67. The file P02_57.xlsx contains data on mortgage loans in 2008 for each state in the United States. The file is different from others in this chapter in that each state has its own sheet with the same data in the same format. Each state sheet breaks down all mortgage applications by loan purpose, applicant race, loan type, outcome, and denial reason (for those that were denied). The question is how a single data set for all states can be created for analysis. The Typical Data Set sheet indicates a simple way of doing this, using the powerful but little-known

INDIRECT function. This sheet is basically a template for bringing in any pieces of data from the state sheets you would like to examine. a. Do whatever it takes to populate the Typical Data

Set sheet with information in the range B7:D11 and B14:D14 (18 variables in all) of each state sheet. Add appropriate labels in row 3, such as Asian Dollar Amount Applied For.

b. Create a table of correlations between these variables. Color yellow all correlations between a given appli- cant race, such as those between Asian Mortgage Application, Asian Dollar Amount Applied For, and Asian Average Income. Comment on the magnitudes of these. Are there any surprises?

c. Create scatterplots of White Dollar Amount Applied For (X axis) versus the similar variable for each of the other five applicant races. Comment on any patterns in these scatterplots, and identify any obvious outliers.

CASE 3.1 Customer Arrivals at Bank98 Bank98 operates a main location and three branch locations in a medium-size city. All four locations perform similar services, and customers typically do business at the location nearest them. The bank has recently had more congestion— longer waiting lines—than it (or its customers) would like. As part of a study to learn the causes of these long lines and to suggest possible solutions, all locations have kept track of customer arrivals during one-hour intervals for the past 10 weeks. All branches are open Monday through Friday from 9 a.m. until 5 p.m. and on Saturday from 9 a.m. until noon. For each location, the file C03_01.xlsx contains the number of customer arrivals during each hour of a 10-week period.

The manager of Bank98 has hired you to make some sense of these data. Specifically, your task is to present charts and/ or tables that indicate how customer traffic into the bank locations varies by day of week and hour of day. There is also interest in whether any daily or hourly patterns you observe are stable across weeks. Although you don’t have full information about the way the bank currently runs its operations—you know only its customer arrival pattern and the fact that it is currently experiencing long lines—you are encouraged to append any suggestions for improving opera- tions, based on your analysis of the data.

CASE 3.2 Saving, Spending, and Social Climbing The best-selling book The Millionaire Next Door by Thomas J. Stanley and William D. Danko (Longstreet Press, 1996) presents some very interesting data on the characteristics of millionaires. We tend to believe that people with expen- sive houses, expensive cars, expensive clothes, country club memberships, and other outward indications of wealth are the millionaires. The authors define wealth, however, in terms of savings and investments, not consumer items. In this sense, they argue that people with a lot of expensive things and even large incomes often have surprisingly little wealth. These people tend to spend much of what they make on consumer items, often trying to keep up with, or impress, their peers.

In contrast, the real millionaires, in terms of savings and investments, frequently come from “unglamorous”

professions (particularly teaching), own unpretentious homes and cars, dress in inexpensive clothes, and otherwise lead rather ordinary lives.

Consider the (fictional) data in the file C03_02.xlsx. For several hundred couples, it lists their education level, their annual combined salary, the market value of their home and cars, the amount of savings they have accumulated (in sav- ings accounts, stocks, retirement accounts, and so on), and a self-reported “social climber index” on a scale of 1 to 10 (with 1 being very unconcerned about social status and material items and 10 being very concerned about these). Prepare a report based on these data, supported by relevant charts and/or tables, that could be used in a book such as The Millionaire Next Door. Your conclusions can either support or contradict those of Stanley and Danko.

09953_ch03_ptg01_084-131.indd 130 04/03/19 12:10 PM

3-6 Conclusion 1 3 1

CASE 3.3 Churn in the Cellular Phone Market The term churn is very important to managers in the cellu- lar phone business. Churning occurs when a customer stops using one company’s service and switches to another com- pany’s service. Obviously, managers try to keep churning to a minimum, not only by offering the best possible service, but by trying to identify conditions that lead to churning and taking steps to stop churning before it occurs. For exam- ple, if a company learns that customers tend to churn at the end of their two-year contract, they could offer customers an incentive to stay a month or two before the end of their two-year contract. The file C03_03.xlsx contains data on

over 2000 customers of a particular cellular phone company. Each row contains the activity of a particular customer for a given time period, and the last column indicates whether the customer churned during this time period. Use the tools in this chapter (and possibly the previous chapter) to learn (1) how these variables are distributed, (2) how the variables in columns B–R are related to each other, and (3) how the variables in columns B–R are related to the Churn variable in column S. Write a short report of your findings, including any recommendations you would make to the company to reduce churn.

CASE 3.4 Southwest Border Apprehensions and Unemployment

Illegal immigration across the southwest border of the United States has been a hotly debated topic in recent years. Many people argue that illegal immigration is related to the U.S. unemployment rate: when the unemployment rate is up, there are more attempts to cross the border illegally. The file C03_04.xlsx contains data on the number of apprehensions

on the southwest border and the U.S. unemployment rate. There are both yearly data since 1981 and monthly data since 2000. Using these data, make an argument, one way or the other, for whether apprehensions are related to the unem- ployment rate.

APPENDIX Using StatTools to Find Relationships The StatTools add-in from Palisade can be used to speed up several procedures discussed in this chapter:

• It can calculate a table of summary measures of a numeric variable, broken down by a categorical variable. (This is illustrated in the Introduction to StatTools video that accompanies the book.)

• It can create one or more scatterplots from its list of Sum- mary Graphs on the StatTools ribbon. It can even color the

points in a scatterplot based on the category of a categorical variable.

• It can create a table of correlations (or covariances) from the Summary Statistics dropdown list on the StatTools ribbon.

Note, however, that StatTools doesn’t do pivot tables. There is no need because Excel does them very nicely.

09953_ch03_ptg01_084-131.indd 131 04/03/19 12:10 PM

CHAPTER 4 Business Intelligence (BI) Tools for Data Analysis

INTERNATIONAL CHART DAY Charts have been an important part of data analysis for well over a century, but with the increasing emphasis on data visualization, there is now an official International Chart Day as of April 26, 2018. It is sponsored by Tumblr (software) and the Society for New Design in collaboration with the Office of United States Representative Mark Takano from California. In their website at https://www.internationalchartday.com/, they describe their purpose as follows:

Charts and other variations of information graphics have been around for hundreds of years, and are an important tool for making complex information easier to understand. How- ever, not all charts are created equal: they can sometimes be too complicated or convey false and/or misleading informa- tion if not executed correctly.

On April 26, chart lovers will unite to celebrate the first-ever “International Chart Day,” an opportunity for the infographic creating community to engage the public by shar- ing their favorite examples, further explaining the history and value of charts, and sharing insight in to how to create high-quality, visually engaging, informative visualizations.

The goal is to assist the public in becoming better consumers of data, information and news.

TO THIS END WE SHALL:

1. Celebrate charts and infographics of all types; 2. Help the public understand how to read charts and gain useful insights from them; 3. Help chart makers of all levels understand the necessary components of a truthful

chart; 4. Encourage the wider usage and adoption of charts; 5. Combat the spread of fake news by making the public smarter consumers of

information.

The organizers of the first International Chart Day invited everyone to share their favorite data visualizations online. Therefore, Tableau Software, the developer of Tableau Public discussed in this chapter, shared five of its recent “Viz of the Day” winners. (See the Blog section at https://public.tableau.com/s/.) These five illustrate the immense variety of data visualizations:

1. Women’s Representation in Politics, by Yvette Kovacs of Starschema in Budapest, Hungary, shows the degree to which women are represented in parliaments around the world. She uses the same small chart type for each country, listed in decreasing order of women’s representation and color coded to show women’s representation in parliament. When you hover over any of these, you see a line chart of women’s representation through time in that country.

2. The Best Way to Get Around Austin TX, by Xeomatrix, is a multi-faceted data dash- board that shows the hotspots for bike share usage in Austin by time of day. The dash- board contains three charts and a filter pane on the left. The first chart is a map of different bicycle start stations with bubbles sized and colored to reflect the number of

N u rP

h o to

/G e tt y

Im a g

e s

09953_ch04_ptg01_132-182.indd 132 04/03/19 12:18 PM

4-1 Introduction 1 3 3

trips from that station. The second chart is a bar chart of the trips to the most popular bicycle end stations. The third chart is a colored-coded grid showing the number of trips by day and hour of day.

3. Ratios of Inequity, by Nai Louza of UBS in London, uses a map to show the states in the United States and Puerto Rico with the highest and lowest ratios of high- income households to low-income households. The map uses a color-coded (but same-sized) hexagon for each state, along with a few text boxes to add commentary. When you hover over any state’s hexagon, you see a line chart of how the inequity ratio has changed over time. (As of 2016, New Jersey had the highest inequity ratio, 1.97, and Puerto Rico had the lowest, 0.02.)

4. Explore European Cities on a Budget, by Sarah Bartlett of Slalom Consulting, uses a graphics-packed visualization to describe the five cheapest European cities to visit: Kiev, Krakow, Belgrade, Bucharest, and Sofia. It begins with a bar chart of dollars per day for these cities. Then it has a separate section for each of the five cities, showing costs broken down for accommodations, attractions, meals, drinks, and transportation, as well rainy days per month, some city facts, and a link to explore the city further.

5. The State of the World’s Children, by Simon Beaumont of UNICEF, uses a world map in the top section where you can select a country and a health theme (Adolescents, Basic Indicators, Child Protection, Demographics, Economics, Education, Health, HIV AIDS, Nutrition, Residence, and Women). Then in the bottom section, you can see how the selected country compares to regions of the world on different components of the selected health theme. For example, if you choose United States and Demograph- ics, you can see how the United States compares to Western Europe on life expectancy. (Western Europe is slightly ahead.)

It is unlikely that International Chart Day will become another Fourth of July, but its significance is that it underscores the importance of data visualization in today’s world.1 The visualizations described here provide insights, tell stories, and can be addictive. Fortu- nately, as you will see later in this chapter, the visualization software now available allows everyone to display data in creative ways.

4-1 Introduction As data analysis has become more important in the business world, Microsoft and other software companies have created better tools for analyzing data—or, to use a popular term, for creating business intelligence (BI). This chapter discusses some of these tools, tools that didn’t even exist 10 or 15 years ago. Our primary focus is on a set of Microsoft add- ins for Excel sometimes called the Power BI suite. They include Power Query and Power Pivot, which we will discuss in some detail, plus two others, Power View and Power Map, that aren’t discussed here. Microsoft refers to the role of these add-ins as self-service BI. The meaning is that the people who need to analyze data for their jobs—financial analysts, marketing analysts, cost accountants, and so on—no longer need to rely on IT departments (and the long wait time this often requires). Instead, they can use these “power” tools them- selves to get the results they need—right away. The power tools for Excel are not the only self-service BI tools Microsoft offers. Many companies use the analysis tools in Microsoft’s SQL Service database software, called SQL Service Analysis Services (SSAS), and Micro- soft also offers a stand-alone package called Power BI. The latter has tools like those we will discuss for Excel, but it is a separate package. In addition, other software companies offer similar BI software; the term “BI” is not used only by Microsoft.

When you think of data analysis, you probably think of numbers, whether they reside in a large table or a summary report. However, we humans tend to gain more insights when we see graphs of the numbers. Several types of graphs, including histograms,

1 For an elementary but insightful discussion of the difference between effective and ineffective visualizations, we recommend the article by Camm et al. (2017).

09953_ch04_ptg01_132-182.indd 133 04/03/19 12:18 PM

1 3 4 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

scatterplots, and others illustrated in the previous two chapters, have been used for over a century. Recently, however, there has been an emphasis on data visualization, where data are shown graphically in new and imaginative ways. We discuss a popular non-Microsoft software package, Tableau Public. We chose to include Tableau Public for three reasons: It is powerful, it is quite easy to learn, and it is free.

Finally, you cannot always assume that the data you obtain are clean. There can be—and often are—many instances of wrong values, which can occur for many reasons. Unless you fix these at the beginning, the resulting data analysis can be seriously flawed. Therefore, we conclude this chapter by discussing a few techniques for cleansing data.

4-2 Importing Data into Excel with Power Query Excel is not the only software that can be used to analyze data, but it is almost certainly the most popular. Unfortunately, the data sets to be analyzed are often stored somewhere else, not in Excel workbooks. Therefore, if you want to use Excel for data analysis, you must learn methods for importing data into Excel. This is not a new topic. Technologies for importing external data into Excel have existed for years, and they continue to evolve. The good news is that the most recent tools are better and easier to use than ever. This section illustrates these tools, often referred to as Power Query.2 Be aware, however, that this section is aimed at Excel for Windows users (or Mac users with Windows emulation software). The Power Query tools have not yet been included in Excel for Mac, and we have no way of knowing when they will be. (A Web search will quickly show that Mac users are clamoring for these tools, so hopefully Microsoft is listening.) Excel for Mac does have some tools for importing data, but they are older tools and will be discussed here only briefly.

4-2a Introduction to Relational Databases The data sets discussed in the previous two chapters can also be called databases, but they are a special type of database, a single-table database. Traditionally, these have been called flat files. Many of the databases you analyze will indeed be single-table databases, where all the data are held in a single rectangular table. More generally, however, a database is a set of related tables, where each table is a data set of the type discussed in the previous two chapters. Because the tables are related, this type of database is called a relational database.

This raises two questions: (1) Why are data often stored in multiple tables rather in a single large table, and (2) how are the tables related? The answer to the first question is to avoid redundancy. Suppose you want to create a database of donors and donations to a charitable organization. For the donors, you want to store their name, their address, their email address, their phone number, and possibly more. For the donations, you want to store the donor’s name, the date of the donation, the amount, and possibly more. Now imagine that Carol Wilkinson is a frequent donor. If you store everything in a single table, there will be a row for each donation, and each such row for Carol will repeat her address, email address, phone number, and possibly other personal information. Not only does this inflate the size of the database, but it introduces the possibility of errors. If Carol moves to a new home, you must then be careful to change her address in each of her donation rows. This is not a good way to manage data.

The solution is to split the donors and donations into separate tables. The Donors table will include a single row for each person who has ever donated. All addresses, email addresses, and phone numbers will be listed in this table exactly once, making them easy to modify if someone moves, for example. The Donations table will include a sin- gle row for each donation, including the date and amount of the donation and possibly

Power Query refers to Excel’s improved tools for importing data from various sources into Excel.

2 There is no mention of the term Power Query in any Excel ribbon, but the term is still used in discussions of Excel’s newest query tools.

09953_ch04_ptg01_132-182.indd 134 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 3 5

other information. The result is exactly what is required: two relatively small tables with no redundant information.

But this brings us to the second question: How are the tables related? For example, if you want information about the donor who donated $1500 on May 21, 2018, how do you know which donor this is? The answer is to use keys. In the Donors table, there will be a primary key column that is a unique identifier of the donor.3 There are multiple ways to create primary keys. One possibility is to use an existing unique identifier, such as a social security number. Another possibility is to use a last name with a numeric suffix, such as wilkinson1. (Then the next Wilkinson donor would be wilkinson2, and so on.) Finally, it is always possible to use an autonumber key, where the first donor has key 1, the second has key 2, and so on. Then if there are already 783 donors in the database and another is added, that donor’s key is automatically 784. It really doesn’t matter what primary key is used, so long as it is unique for each donor.

Now suppose an autonumber primary key is used, and Carol’s key is 241. This is listed in the Donors table. Then a foreign key column is appended to the Donations table. Each time Carol donates, this foreign key value will be 241. The actual number has no intrinsic meaning; it is used only for lookup purposes. If you want to see all Carol’s donations, you go to the Donations table and search for foreign key 241. Note that a foreign key is not unique. Carol can make several donations, so her foreign key 241 can appear multiple times in the Donations table. In the other direction, if you see a donation with foreign key 137, you can search the Donors table for the unique primary key 137 to see who made this donation.

The actual values in primary and key fields are often meaningless, but they serve their purpose of relating tables.

3 As we have mentioned before, the term “field” is often used instead of “column,” especially in database discussions. We will use the two terms interchangeably.

The whole purpose of separating data into separate tables in a relational database is to avoid redundancy. Each table contains data about a specific entity, such as customers, products, orders, and so on. However, primary and foreign keys are then necessary to relate the tables. For example, these keys let you look up infor- mation about all orders made by a specific customer for a specific product.

Fundamental Insight

The relationship between the Donors and Donations table is called a one-to-many relationship. Each donation corresponds to exactly one donor, but one donor can make many donations. In this case, the primary key resides in the “one” table, and the foreign key resides in the “many” table.

Tables in a relational database are related through primary and foreign keys. A primary key is a unique identifier, whereas a foreign key is used for lookup pur- poses and is not necessarily unique. A primary key resides in the “one” side of a one-to-many relationship; a foreign key resides in the “many” side.

There can also be many-to-many relationships. A typical example is of orders and products from an online store. A specific order can be for many products, and a specific product can be part of many orders. Then there will be an Orders table and a Products table, each with a primary key. The usual way to relate them is with a linking table (also called a bridge table) that might be called Order Details. This will be a long thin table with a row for each product in each order. For example, if an order is for five products, this will add five rows to the Order Details table. This linking table will have at least two foreign

09953_ch04_ptg01_132-182.indd 135 04/03/19 12:18 PM

1 3 6 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

keys, one for the order and one for the product, and possibly nothing else. These keys let you look up all orders that contain a specific product or all products in a specific order. Note that the relationship between Orders and Order Details is one-to-many, as is the rela- tionship between Products and Order Details.

Tables and relationships between them are usually shown in a relationships diagram. The relationship between Donors and Donations appears in Figure 4.1. The relationship between Orders, Order Details, and Products appears in Figure 4.2. In each case, the “ID” fields are the key fields. Key fields do not have to have “ID” in their names, and primary/ foreign key pairs do not even have to have the same names, but these customs are often used. Note the “1” next to the “one” side of each relationship and “*” next to the “many” side. (These diagrams are from Power Pivot, discussed later in the chapter. Diagrams in other software packages often use the infinity symbol, ∞, for the many side.)

Until recently, all multi-table relational databases were stored in a relational data- base management system (RDBMS). This is software that creates and manages databases. Microsoft Access, part of the Office suite, is one such system. Access is a desktop system where your database files can reside locally on your hard drive. (They have an extension .mdb or .accdb.) Server-based systems are more common in corporate settings, where the databases are stored on a server and are usually managed by a database administrator (DBA). To access the data, you typically need to log in with appropriate credentials. Some of the most popular server-based systems are Microsoft SQL Server, Oracle Database, IBM Db2, and MySQL.

Data are usually stored in a RDBMS, but data analysts typically want to perform their analyses in Excel. Therefore, several technologies have been developed in the past few decades to import data from a RDBMS into Excel. All these technologies involve one or more queries, where a query specifies exactly which data you want to import. At the very least, the query must specify the table or tables that contain the data you want, the columns you want, and the rows you want. Queries are so common that a language for queries was developed about 30 years ago. This language is called Structured Query Language (SQL), and it continues to be used today. The power of SQL is that, with minor modifi- cations, it can be used to query any RDBMS. However, it presents a hurdle for the many Excel users who do not know SQL. Therefore, Microsoft and other companies have been trying to make querying easier by creating more transparent tools. In Microsoft’s case,

SQL is the traditional language for specifying a query.

Figure 4.1 Relationship between Donors and Donations

Figure 4.2 Relationship between Orders, Order Details, and Products

09953_ch04_ptg01_132-182.indd 136 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 3 7

the result is Power Query, as we will discuss shortly. However, it is useful to discuss other possibilities.

First, it has always been possible to create multi-table relational databases directly in Excel. Each table is of the type discussed in the previous two chapters. They could all be in the same worksheet or in separate worksheets. To be relatable, they must have primary and foreign key columns as discussed earlier. Then to relate them, you can use VLOOKUP functions. Figure 4.3 illustrates a simple example, where the primary key in column A is an autonumber key, and the formula in cell K4, copied down, is =VLOOKUP(G12,$A$3:$E$10,2). (You can find this example in the Multi-Table Query in Excel Finished.xlsx file.) This VLOOKUP method is frequently used, but it is cumbersome. Besides, the tables involved might be too large to fit into Excel’s (approximately) one-million row limit.

Figure 4.3 Multi-Table Query in an Excel Worksheet

DonationDateName

1 2 3 4 5 6 7

8 9

A B C D E F G H I J K L M

10 11 12 13 14 15 16 17 18 19 20

Bob Hanson Cheryl Jones Tim Jenkins Pat McCain Don Lemon Bill Paxon Candy Entson

DonorID 1 7 5 2 4 6 1 4 7 3 7 2 3 4 3 1 3

Date 01/22/18 01/28/18 02/15/18 03/11/18 04/11/18 04/22/18 05/16/18 05/26/18 06/01/18 06/20/18 08/11/18 08/12/18 08/14/18 09/11/18 09/16/18 09/17/18 10/02/18

Donation $525 $475

$25 $1,100

$550 $250 $325

$50 $200 $900

$25 $650 $950 $975 $650

$1,175 $150

DonorID Name Other info... Candy Entson Tim Jenkins Candy Entson Cheryl Jones Tim Jenkins Pat McCain Tim Jenkins Bob Hanson Tim Jenkins

06/01/18 06/20/18 08/11/18 08/12/18 08/14/18 09/11/18 09/16/18 09/17/18 10/02/18

$200 $900

$25 $650 $950 $975 $650

$1,175 $150

Donors table Donations table Query: donations after May

Second, the Power Query tools are only the latest in the evolution of Microsoft’s tools for importing data into Excel. Depending on your version of Excel, you might have to use older (and less friendly) tools. Figure 4.4 shows the tools in Excel 2010 for Windows. They are in the Get External Data group on the Data ribbon.

Here are a few comments about these 2010 tools:

• The From Access tool allows you to import data from one table only or from a saved query in Access. The latter is useful, but it means you must first create a query within Access.

• The From Web tool allows you to import data from the Web, but it doesn’t work on many Web data sets. The similar Web tool in Power Query, discussed later in this section, is much better.

• The From SQL Server and From Analysis Services tools require you to have a SQL Server account.

• The From Microsoft Query tool was probably the most popular query tool in Excel before Power Query. It still works fine, but Power Query is more powerful and easier to use.

Excel 2016 for Mac (at least as of this writing) is another story altogether. It has the Get External Data tools shown in Figure 4.5. The New Database Query has only two items in its dropdown list: SQL Server ODBC and From Database. The former requires a SQL Server account. The latter returns an alert that you must first install an ODBC Driver Man- ager, and it provides almost no guidance on how to do so. Also, it is not at all clear how the From HTML tool works. (Do a Web search for this HTML tool for the Mac. No one seems

Excel’s older tools, before Power Query, are still avail- able, but they are not nearly as good as the newer tools.

09953_ch04_ptg01_132-182.indd 137 04/03/19 12:18 PM

1 3 8 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

to understand how it works.) Let’s just say that until improvements are made, Excel for Mac is not set up for importing external data.

From here on, the discussion and screenshots will focus on Excel that is part of Office 365 for Windows, which contains all the Power BI tools, including Power Query. (Even this isn’t quite true. See the following box.)

Figure 4.4 Get External Data Tools in Excel 2010 for Windows

Figure 4.5 Get External Data Tools in Excel 2016 for Mac

It is virtually impossible to keep track of all the versions of Excel and which versions contain which features. Besides, subscribers to Office 365 get monthly updates, often with new features. Excel’s Power BI tools are included in most versions of Office 365 for Windows, but not all. See https://powerpivotpro .com/2015/10/what-versions-of-office-2016-contain-power-pivot/ for details. Also, these tools were available in one form or another in Excel 2013 and even Excel 2010. (They still aren’t available in Excel for Mac.) There is not enough space here to list all the differences, and such a discussion would be hopelessly confusing.4 If you are using a pre-2016 version of Excel for Windows, perform a Web search for, say, “Power Query Excel 2013” or “Power BI Excel 2010.” Eventually, we expect that almost all Excel users will have an Office 365 sub- scription, so that the differences between old versions of Excel will be irrelevant. We also expect that the Power BI tools will eventually be included in Excel for Mac, but there is no way to tell when this will occur.

4 When we tried to learn Power Query in 2016, we bought a book on the subject that was published less than a year before. Even so, most of its explanations and screenshots were already out of date. This is common in today’s software world, so get used to it!

09953_ch04_ptg01_132-182.indd 138 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 3 9

4-2b Excel’s Data Model When you run a query to import external data into Excel, you must decide where you want to store the data. Until recently, there were two basic options: (1) into an Excel table, as discussed in Chapter 2, or (2) into a pivot table (or pivot chart) report, as discussed in Chapter 3. The first option is often a good one unless the size of the imported data is larger than the size of an Excel worksheet—which is sometimes the case. The second option makes sense if your eventual goal is to analyze the data with pivot tables or pivot charts. In this case, the data won’t be directly visible but will instead be stored in a pivot cache, the source for pivot tables and pivot charts.

There is now a third option, a Data Model in Excel. A Data Model essentially mimics the behavior of a RDBMS, but all within Excel. A Data Model stores related tables in a special type of compressed format—not in an Excel worksheet—for use in creating pivot table reports. This is extremely important. It means that you can store your data, even millions of rows of data, inside an Excel workbook and then use Excel’s powerful tools to analyze the data. No VLOOKUPs are necessary, and once the data are in Excel, there is no further need for Access or any other RDBMS. However, the Data Model can keep a link to the source data set, so that if the source data set changes, you can refresh your Data Model with the newest data.

Before Excel’s relatively new Data Model, relational data had to be imported from Access or other relational database packages into Excel for analysis. Now relational databases can be stored in Excel directly, where they can then be ana- lyzed. In short, a Data Model is a relational database that is stored in Excel.

Fundamental Insight

The following example illustrates one way to create a Data Model in Excel.

EXAMPLE

4.1 IMPORTING RELATED DATABASE TABLES The Contoso database stored in the file Contoso Sales.accdb is a “practice” Access database Microsoft created a few years ago. It contains 2007–2009 sales data from the fictional Contoso Company in five related tables: Dates, Products, Product- Categories, ProductSubcategories, and Sales. For illustration, there are two other files, Geography.csv and Stores.xlsx. The Stores file contains data on the stores where Contoso sells its products, and the Geography file contains data (for mapping purposes) about the locations of the stores. These latter two files aren’t currently related to one another or to the Contoso data, but the files contain key fields that can be used to relate them. How can data from Contoso be imported into a Data Model in Excel?

Objective To use Power Query to import the Contoso tables and the Stores and Geography files into a Data Model in Excel.

Solution The following steps can be used to obtain the results in the Contoso Import Finished.xlsx file. (We again warn you that these are based on Excel in Office 365 for Windows.)

1. Open a new workbook in Excel. If the Developer tab isn’t visible, right-click any ribbon, select Customize the Ribbon, and check the Developer item in the right pane of the resulting dialog box. The Developer ribbon has useful tools for pro- grammers, but it also provides an easy way to get to Excel’s add-ins lists.

09953_ch04_ptg01_132-182.indd 139 04/03/19 12:18 PM

1 4 0 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

2. Click COM Add-Ins on the Developer ribbon and check the Microsoft Power Pivot for Excel item. This creates a Power Pivot tab and associated ribbon. You will need Power Pivot to “see” the imported data in the following steps.

3. On the Data ribbon, click Get Data, then From Database, and then From Microsoft Access Database. Maneuver to the Contoso.accdb file and click Import.

4. This opens the Navigator window in Figure 4.6. Fill it in as shown, checking the Select multiple items option and check- ing all five tables. When any table is selected, you see a preview of its data in the right pane.

Power Pivot must be loaded to “see” the data in a Data Model.

Figure 4.6 Navigator Window

5. From the Load dropdown list, select Load To. This opens the Import Data dialog box in Figure 4.7. Fill it out as shown, selecting Only Create Connection and checking the bottom Data Model option, and click OK.

6. A Query & Connections pane with five queries, one for each table, will open in Excel, but when the data have finished downloading, you won’t see any data. Read on.

7. Because you filled out the Import Data dialog box as shown in Figure 4.7, the data don’t appear in an Excel table or pivot table. Instead, they are stored in a Data Model. To see the Data Model, select Manage from the Power Pivot ribbon. This opens the Power Pivot window in Figure 4.8, where you can see the imported data. This looks like an Excel workbook, with a tab for each of the five tables, but it is quite different, as we will discuss shortly. Note the two buttons circled at the bottom right. These are toggles between data and diagram views. Figure 4.8 shows the data view.

09953_ch04_ptg01_132-182.indd 140 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 4 1

Figure 4.7 Import Data Dialog Box

Figure 4.8 Power Pivot Window: Data Model in Data View

5 We’re not sure why Power Pivot doesn’t “inherit” the relations from the Access file. It does for some Access databases but not for this one. In any case, it’s easy to create the relationships, as explained here.

8. Click the diagram button at the bottom right and then move/resize the tables appropriately to see the relationships diagram in Figure 4.9. Although these tables are related in the Access source file through primary and foreign keys, they aren’t automatically linked with the arrows shown in the figure.5 However, you can drag from the obvious key fields to create these arrows and hence relate the tables. For example, drag from DateKey in the Dates table to DateKey in the Sales table to relate these two tables.

9. Back in the Excel window, click From Text/CSV on the Data ribbon, maneuver to the Geography.csv file, click Load To, and again fill out the dialog box as in Figure 4.7. Then repeat with the Stores data. This time, select From File from the Get Data dropdown list, choose From Workbook, and maneuver to the Sales.xlsx file. Once again, fill out the dialog box as in Figure 4.7.

09953_ch04_ptg01_132-182.indd 141 04/03/19 12:18 PM

1 4 2 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

10. Open diagram view in the Power Pivot window. You will see the new Geography and Stores tables, but they won’t be related to anything. To relate them, drag StoreKey in the Sales table to StoreKey in the Stores table, and then drag GeographyKey in the Stores table to GeographyKey in the Geography table. The final relationships diagram should look like Figure 4.10. (The exact placement of the tables in this diagram is irrelevant. We try to avoid overlapping arrows.)

Figure 4.9 Power Pivot Window: Data Model in Diagram View

Date Table

Most Data Models will include a separate Dates table, even if dates of transactions are already listed in another table. The purpose of a Dates table is to list all possible dates in many possible ways. For example, the Dates table for the date 6/19/2018 might list the year, 2018; the quarter, Q2; the month number, 6; the month name, June; the day of the week, Tuesday; whether it is a weekday, Yes; whether it is a holiday, No; and possibly others. The rea- son is that you’re looking ahead to pivot tables, where you might want to break down transactions in many types of date-related ways. By creating all these date fields right away, you save yourself a lot of work later.

Data Model Tip

You have just seen how a small subset of the Power Query tools can be used to import related data into an Excel Data Model, where they can then be analyzed with pivot tables. If you compare the size of the original blank Excel file with the size of the finished file, you will see that the size increased from about 8 KB to about 2.5 MB. Although you can’t see the data in any Excel worksheets, the data are stored with the workbook in the compressed Data Model format.

Relationships can be created directly by dragging in diagram view of the Power Pivot window.

09953_ch04_ptg01_132-182.indd 142 04/03/19 12:18 PM

4-2 Importing Data into Excel with Power Query 1 4 3

By the way, suppose you select the Table option in Figure 4.7 and you check the Data Model option. This inflates your Excel workbook for no good reason. Each of the five Con- toso tables is imported into a separate worksheet, and the same Data Model is created as before. Therefore, the same data are being stored twice, once in worksheets and once in the Data Model. It’s best to store the data just once, and the Data Model is often the better

choice. Also, as you will see shortly, it doesn’t make sense to select the Pivot Table (or Pivot Chart) option and check the Data Model option. If you eventually want pivot tables, you can create them directly from the Data Model.

Figure 4.10 Final Contoso Relationships Diagram

Importing multiple related tables is just one way of creating an Excel Data Model. The following example illustrates another possibility.

EXAMPLE

4.2 CREATING A DATA MODEL WITH A PIVOT TABLE The file Baseball Salaries with Team Info1.xlsx is similar to the file used in Chapter 2, but now there is only one Salaries sheet (for 2018 players), and there is an extra Teams sheet. The tables on the two sheets have been designated as Excel tables, with table names Salary_Table and Team_Table. The Team table has a row for each of the 30 teams, listing the team’s name, the team’s division (such as National East), and the team’s league (American or National). The Salary table has a row for each player, which includes the player’s team, his position, and his salary. In database terms, the Team field is a primary key for the Teams table and a foreign key for the Salaries table. However, these tables are not yet related; they haven’t yet been linked through their key fields. How can you use a pivot table to create a Data Model from the Teams and Salaries tables?

There is no need to store data in worksheets if the data are already stored in a Data Model.

09953_ch04_ptg01_132-182.indd 143 04/03/19 8:36 PM

1 4 4 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

Objective To create a Data Model by creating a pivot table from the baseball data and, in the process, to relate the two tables.

Solution The following steps are required. (You can see the results in the Baseball Salaries with Team Info1 Finished.xlsx file.)

1. Open the Baseball Salaries with Team Info1.xlsx file. 2. Select any cell in the Salaries table; select PivotTable on the Insert ribbon; fill out the resulting dialog box as shown in

Figure 4.11, making sure to check the Data Model option near the bottom; and click OK.

Figure 4.11 Pivot Table Dialog Box with Data Model Option Checked

3. As usual, a new sheet with a blank pivot table will open, but something else happens. If you open the Power Pivot window as in Example 4.1, you will see that the table from the Salaries sheet has been added to a new Data Model.

4. Back in the pivot table sheet in Excel, drag Salary to the Values area. 5. At this point, the “Active” link at the top of the PivotTable Fields pane is active (bold-

faced), and only the fields in the Salary_Table are listed. But now click the “All” link. All fields in both tables are then listed. (See Figure 4.12.)

6. Expand the Team_Table list and check the Division field, the goal being to find the sum of salaries in each of the six divi- sions. However, because the two tables are not yet related, you will see the warning in yellow in Figure 4.12. Also, the pivot table values will be wrong because Excel is unable to combine data from two tables that aren’t related.

7. To fix the problem, click the Auto-Detect button. In this case, Excel has the intelligence to recognize that the Team fields in the two tables serve as key fields, so it uses them to relate the tables. Nevertheless, it lets you examine the relationship. To do this, click the Manage Relationships button on the resulting dialog box and then click Edit to see the dialog box in Figure 4.13. This has indeed been filled in correctly, and it is exactly how you would fill it in if you clicked the CREATE button instead of the Auto-Detect button in Figure 4.12. The point is that the two tables are now related based on their Team key fields—and the pivot table values are now correct.

If you drag fields from unre- lated tables to a pivot table, you will be warned about a missing relationship.

You can start a Data Model by creating a pivot table from existing worksheet data.

09953_ch04_ptg01_132-182.indd 144 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 4 5

8. To see what else has happened, open the Power Pivot window again. In diagram view, you will see both tables, linked by their key fields. That is, both tables are now part of the workbook’s Data Model. The only difference between this exam- ple and the previous Contoso example is that the Data Model has now been created through pivot table operations, not through importing external data.

9. Want a surprise? To be safe, save your work. Then delete the Salaries and Teams sheets. Now manipulate the pivot table any way you like. The pivot table will still work fine! This is because the pivot table’s source is the Data Model, and the original data sheets are no longer necessary.

Figure 4.12 PivotTable Field Pane with Multiple Tables

Figure 4.13 Edit Relationship Dialog Box

The title of this subsection is Excel’s Data Model, but the two examples have discussed pivot tables and the Power Pivot window, the subjects of Section 4-3. It is difficult to disentangle the Data Model and Power Pivot, and many users evidently think they are the same thing. They are not. The purpose of a Data Model is to store related tables in a com- pressed format so that relational databases that have typically been stored in a RDBMS can now be stored in Excel. Power Pivot provides a window for viewing and managing the data in a Data Model, and it contains advanced tools, some discussed in Section 4-3, that go well beyond Excel’s “regular” pivot table functionality. In short, the Data Model and Power Pivot are not the same, but they work hand in hand.

09953_ch04_ptg01_132-182.indd 145 04/03/19 12:18 PM

1 4 6 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

4-2c Creating and Editing Queries Now we return to querying with Power Query. The tools to do so are in the Get & Trans- form Data group on the Data ribbon, as shown in Figure 4.14. (Again, we remind you that this is for Excel in Office 365 Windows.)

Figure 4.14 Get & Transform Data Group

There are three important strengths of Power Query relative to previous Excel importing tools:

1. You can import data from more sources than before. The Get Data dropdown list in Figure 4.14 shows some of these sources, including the many RDBMS sources, and the three but- tons to the right of Get Data show that you can import data from a text file, the Web, and a table/range in Excel. The list of sources will almost certainly grow in the future.

2. Once you locate data of interest, you can use the new Query Editor to “shape” the data to be imported. For example, you can specify the columns and rows of interest. The Query Editor provides a friendly graphical user interface for building a query, replac- ing the need for a complex SQL statement.

3. Your queries are stored in step-by-step fashion so that they can be modified and reused later. This is important if you make frequent queries from the same basic data source. It means you don’t have start over with each query.

A complete discussion of Power Query’s feature would require an entire book, so we will only lead you through two examples to illustrate some possibilities. Be aware that the query tools continue to evolve, so the details shown here may have changed by the time you read this. (Try the following: From Excel’s File menu, click Feedback and then the Send a Suggestion button. This opens a Web page of suggestions by users. If you scroll through the list, you will see quite a few about Power Query. This is a very popular tool, and users want it to be even better.)

The Query Editor is essen- tially a user-friendly substi- tute for the SQL query language. It lets you choose the rows and columns to include—and much more.

09953_ch04_ptg01_132-182.indd 146 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 4 7

Figure 4.15 Query Editor

For example, if you click the Choose Columns button, you see a list of all columns in the selected table, and you can decide which columns you want to keep. As for rows, the dropdown arrow next to each column opens a window, just like you saw for tables in Chapter 2, where you can filter the rows to be returned. You can also transform the data in various ways. For example, the Split Column button lets you separate names like Wilson, John into two separate columns, based on the comma delimiter. For advanced users, the Advanced Editor button lets you see and manipulate the query command used behind the scenes to retrieve the data. (This is written a new query language called M, not SQL.) As you choose columns, filter rows, and so on, watch the Applied Steps to the right. All your actions are “recorded” as steps in the query, and you can later modify any of these steps as necessary.

EXAMPLE

4.1 IMPORTING RELATED DATABASE TABLES (CONTINUED) When you imported the Contoso data earlier, you imported all of the five selected tables, that is, all the rows and all the columns. What other importing options does Power Query provide?

Objective To learn how to “shape” a query with Power Query.

Solution If you select From Microsoft Access Database in Figure 4.14 and then maneuver to the Contoso.accdb file, you get the Navigator window in Figure 4.6, where you can select the tables of interest. For now, select all five tables as before. However, instead of loading the data right way, click the Edit button. This opens the Query Editor in Figure 4.15. This lets you preview the data, but, more importantly, it lets you “shape” the data to be imported.

09953_ch04_ptg01_132-182.indd 147 04/03/19 12:18 PM

1 4 8 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

This just scratches the surface of the Query Editor. We haven’t discussed all the buttons on the its Home ribbon, and we haven’t even shown its Transform, Add Column, and View ribbons. The Query Editor is a very powerful tool. If you plan to perform frequent queries in your eventual job, you will want to master this tool.

When you are finished shaping the data with the Query Editor, click the Close & Load dropdown arrow. This presents two options:

• The Close & Load option uses the current default setting to return the data to Excel in some way. This default setting seems to change, depending on your recent queries, so we don’t recommend it.

• The second option, Close & Load To, is safer. It opens the Import Data dialog box in Figure 4.7, where you can decide how you want the data imported.

For either option, the Queries & Connections pane in Figure 4.16 opens on the right side of the Excel window. (You can then close it or reopen it with the Queries & Connections button, a toggle, on Excel’s Data ribbon.) This pane is very handy. For example, if you dou- ble-click any item, the Query Editor opens, where you can modify your query if necessary.

By double-clicking any query in the Queries & Connection pane, you can return to the Query Editor and modify the query.

Figure 4.16 Queries & Connections Pane

The next example involves a Web query. Obviously, Web pages have a variety of structures, but many have data in some type of tabular form. When you find a Web page with interesting data, you can try to import the data into Excel with the From Web but- ton on the Data ribbon. Not all Web queries will succeed—this depends on how the Web page was created—but many more succeed with Power Query than with Excel’s older Web query tool.

The newer Web query tool is much better—that is, much more likely to succeed— than the original Web query tool.

09953_ch04_ptg01_132-182.indd 148 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 4 9

EXAMPLE

4.3 IMPORTING BASEBALL SALARY DATA FROM THE WEB We first searched the Web for baseball salaries and found promising 2018 salary data at https://www.usatoday.com/sports/mlb /salaries/2018/player/all/. How can you import these data into Excel?

Objective To use Power Query’s new and improved Web query tool to import baseball salary data into Excel.

Solution To import the data into Excel, do the following:

1. Click the From Web button on the Data ribbon, paste this URL into the resulting dialog box, and click OK. 2. The Navigator window in Figure 4.17 opens with two items on the left: Document and Table 0. In general, more items

might be listed, and you can click any of them to see a preview on the right. In this case, Document has nothing interest- ing, but Table 0 appears to have the desired salary data, so select it and click Edit.

Figure 4.17 Navigator Window for Baseball Example

3. The Query Editor open, exactly as in the previous example. At this point, you can delete “junk” columns or rows and eventually use the Close & Load To option to return the data to Excel.

As these two examples illustrate, the basic sequence for all data imports is the following: (1) select the type of data source; (2) choose the tables of interest in the Navigator win- dow; (3) use the Query Editor to shape the query; and (4) use a Close & Load To option to import the data into Excel. In this sense, Power Query is consistent across different types of data sources, and it is quite easy to learn.

As a final note, keep in mind that when you import data from an external source such as an Access file or the Web, the imported data remain linked to the source. This means that if the source data change, you can click the Refresh button on Excel’s Data ribbon to update your imported data. If, for whatever reason, you want to sever the link, you can right-click the query in the Query & Connections pane and select Delete. Your imported data won’t disappear; it just won’t be refreshable.

The data imported with Power Query retain a live link to the source data.

09953_ch04_ptg01_132-182.indd 149 04/03/19 12:18 PM

1 5 0 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

6. Search the Web for NBA Salaries HoopsHype. (NBA stands for National Basketball Association.) Unless the site has changed, you should find a list of all NBA player salaries for the current season and several future seasons. Copy the URL and use it with Power Query to import the salaries for the current season only. (This was the 2017–2018 season as of this writing.) Create a histogram and box plot of these salaries. Is the salary distribution what you’d expect? Explain.

7. The P04_07.txt file is a text file listing the number of mobile subscriptions in four years for several countries. (You can view the data in NotePad if you like.) Use Power Query to import the data into an Excel table, using the Query Editor to delete the first and last columns and the last four rows. The result should be a table with three columns and four rows for each country, one for each year. An alternative arrangement has five columns (Country, 2002, 2003, 2004, 2005) and one row per country. Show that you can create this alternative arrangement with a pivot table based on the imported data.

Level B 8. Visit the website https://finance.yahoo.com/quote/

YHOO/history?p=YHOO (or search the Web for Yahoo historical stock prices if this link no longer works) to find historical stock prices for any specified stock symbol, such as MSFT for Microsoft. Use this website to find all daily stock price data for Microsoft for 2017. Copy the URL and use it with Power Query to import the Microsoft 2017 data into a table in Excel. Delete the rows that have to do with dividends and sort in chronological order. Finally, create a line chart of the adjusted closing prices. (Note: There are two quirks about importing from this Yahoo site. First, Yahoo lets you import only 100 rows at a time. This means you will have to perform multiple queries to get all the 2017 data. Second, it imports the stock price data as text, not numeric, evidently because of the dividend rows. However, you can change this in the Query Editor with the Data Type dropdown on the Transform ribbon.)

9. The P04_09.mdb file is an Access database file for a fic- tional bowling league. This league consists of teams that play matches—one team against another—in various tournaments. Use Power Query to import all the tables into a Data Model. Then use the Power Pivot window to view the data and relationships. There are problems with the relationships, which you would find if you started creating pivot tables—the structure of this database is complex. However, using only the tables themselves in the Power Pivot window, not pivot tables, answer the fol- lowing questions: a. There are 20 tournaments listed in the Tournaments

table. How many of these have been played? b. How many matches are there per tournament?

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. The P04_01.mdb file is an Access database file with

two related tables about classical musicians. Use Power Query to import the data into a Data Model and then cre- ate a pivot table that shows the number of musicians for each instrument, sorted in decreasing order of the count.

2. The P04_02.mdb file is an Access database file with several related tables about food recipes. Use Power Query to import all tables and fields into a Data Model. Using Design view in the Power Pivot window as a guide, indicate the following types of relationships: (1) between Ingredient_Classes and Ingredients, (2) between Ingredients and Recipes, (3) between Measurements and Recipes, and (4) between Recipe_Classes and Recipes. If there are any linking tables, what role do they play? Finally, use a pivot table based on the Data Model to find, for each ingredient, the number of recipes that use that ingredient.

3. The P04_03.xlsx file contains four tables on four sep- arate sheets with information about a company’s sales in the latter part of 2018. These tables are not currently related, but they are relatable through key “ID” fields. Create a pivot table based on any of the tables—you can decide which—and check the option to add this to a Data Model. Then add fields from all the tables to the pivot table, creating relationships when necessary. (That is, mimic Example 4.2.) Manipulate the pivot table to find the Sum of QuantityOrdered, broken down by month of OrderDate and ProductName, but only for products in the Clothing category. Finally, after checking in the Power Pivot window that the tables are related correctly, delete the original four sheets. Your pivot table should still work fine.

4. The P04_04.mdb file is an Access database file with several related tables created by an entertainment agency. Use Power Query to import three of these tables, Entertainers, Entertainer_Members, and Members, into a Data Model. Is the relationship between Entertainers and Members one-to-many? What role does the Entertainer_Members play? Which, if any, of the members are in at least two entertainer groups?

5. The P04_05.mdb file is an Access database file with several related tables on college courses that students just completed. Use Power Query to import all tables and fields into a Data Model. Which of the tables, if any, are linking tables, and why are they required? By examining only the tables in the Power Pivot window (that is, not using pivot tables), find the following information about Steven Buchanan’s schedule: the sections of the courses he took, the rooms and buildings of these sections, the days and times of these sections, and his grades in these sections.

09953_ch04_ptg01_132-182.indd 150 04/03/19 12:18 PM

4-2 Importing Data into excel with power Query 1 5 1

e. The codes in the Code column are 001 to 048 for the contiguous 48 states. The other codes, above 100, are for regions. Delete all rows with these region codes.

f. The State Codes.xlsx file contains a table of codes and the corresponding states. Copy its data some- where to the right of the imported temperature data. Then insert a new column, State, to the right of the temperature data Code column and use a VLOOKUP function to fill it with the state names.

g. Add three new columns to the right of the Year col- umn: Low, High, and Average. Low and High should be the minimum and maximum monthly temperature for that year, respectively, and Average should be the average over all 12 months of that year.

h. Create a pivot table and pivot chart, based on the tem- perature data, where you can see a line chart of the Low, High, and Average values across years for any selected state. Filter out the year 2005 and use a slicer for the State filter.

12. The P04_12.txt file is a text file with data about uninsured people in each state for the years 2003–2007. Use Power Query to import this data into an Excel table (not a Data Model) and sort in increasing order of Year. As you will see, this is a wide table, with four columns for every state and only five rows, and it isn’t very useful for analysis. It would be better to have a narrow, tall table with columns for Year, State, and the four uninsured numbers/percentages reported. Do whatever it takes to rearrange the data this way. Then create a pivot table where you can see any of the numbers/percentages across years for any selected state. (Hint: You could do this with a lot of cutting and pasting. However, a better way is to use Excel’s INDEX function with clever row and column indexes. For example, the number of uninsured females in Alaska in 2004 can be found with =INDEX(range,2,5), where range is the range of the imported numbers/percentages.)

13. You have been hired to create a relational database for a regional soccer league. Each team in the league has players, and the teams play one another, maybe multiple times, during the soccer season. How would you structure this database? Would you use a single-table database? If so, what fields would the table include? Would it be a multi-table database? If so, what tables would be included, how would they be related, and what fields would each table include? As you develop this structure, think about the questions you might want to ask of the database, probably with pivot tables. There is clearly no single correct structure, but there are structures that are better than others.

c. How many games are there per match? d. How many players are on a team? e. Who is the team captain of the Terrapins? f. Where was match 8 played, which teams played in

match 8, and what was the total handicapped score of each of these teams in game 2 of this match?

10. The P04_10.mdb file is an Access database file created by a classical music CD collector (when CDs were “in”). Keep in mind that a classical CD can have multiple works by multiple composers, played or sung by multiple artists of various types or groups. Use Power Query to import all tables and fields into a Data Model. Then browse the tables and view the relationships diagram in the Power Pivot window. E x p l a i n t h e r e l a t i o n s h i p s b e t w e e n t a b l e s , w i t h emphasis of the role of the CDs_Works table. (Move the tables around in Design view to get a clearer picture of the relationships.) Then use a pivot table to find the count of works, by composer, and sort them in decreasing order of count. (Note that these counts will often include different recordings of the same work, such as Beethoven’s Fifth Symphony.)

11. The srn_tmp.txt file is a text file that contains monthly temperature from 1895 through January 2005. The srn_data.txt is a “data dictionary” that explains the temperature file. (The data dictionary mentions two other files, srn_pcp.txt and srn_pdsi.txt. These files have been provided, but they aren’t used here.) Proceed as follows: a. Use Power Query to import the temperature data into

an Excel table (not a Data Model). Use the Query Edi- tor for only one purpose: to change the data type for column 1 to text. (You can do this with the Data Type dropdown on the Transform ribbon.)

b. In Excel, change the column headings for columns 2 to 13 to month names: Jan, Feb, etc. Also, delete the last column, which is empty.

c. Create two new columns to the right of column 1: Code and Year. Fill the Code column with the first three digits of the column 1 value, using Excel’s LEFT function, and fill the Year column with the last four digits of the column 1 value, using Excel’s RIGHT function. Then copy the Code and Year columns, paste them over themselves as values, and delete column 1, which is no longer necessary.

d. Scan down to any year 2005 row. Only January’s tem- perature is listed. The rest are missing, denoted by the code -99.9. Perform a search and replace for -99.9, replacing each with a blank.

09953_ch04_ptg01_132-182.indd 151 04/03/19 12:18 PM

1 5 2 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

4-3 Data Analysis with Power Pivot This section discusses the most publicized member of Microsoft’s Power BI tools, Power Pivot. This add-in was introduced as a free add-in for Excel 2010 and is now an integral part of Excel.6 The first question you probably have is a natural one: With the pivot table functionality discussed in the previous chapter already in Excel—without Power Pivot— why is Power Pivot needed? This is not an easy question to answer, at least not before you have seen what Power Pivot provides. Hopefully, by the end of this section, you will have some appreciation for why many data analysts, those who have used pivot tables for years, love Power Pivot.

We mentioned earlier that although the Data Model and Power Pivot are not the same thing, they work hand in hand for powerful data analysis. To convince you that they are not the same thing, try the following:

1. Click the COM Add-ins button on the Developer ribbon and make sure the Microsoft Power Pivot for Excel item is not checked.

2. Open the result of Example 4.1, where a seven-table Data Model has been created from the Contoso database. You can’t “see” the data, but the seven tables are stored in the Data Model.

3. Select Pivot Table from the Insert ribbon and make sure the Use this workbook’s Data Model option is checked, as shown in Figure 4.18.

The Power Pivot add-in is now part of Excel, but it still must be added in through the COM add-ins list.

6 We remind you again that Power Pivot is not included in all versions of Office 365. You can check your own version by viewing the COM add-ins list on the Developer ribbon. Unless Power Pivot is on the list, your version doesn’t have it.

Figure 4.18 Basing a Pivot Table on the Data Model

4. A blank pivot table will appear, and all fields from all seven imported tables will be available for the pivot table. You can create a few example pivot tables to confirm that everything works correctly.

The point here is that you do not need Power Pivot to create pivot tables from an existing Data Model. Also, as you saw in the previous section, Power Pivot is not needed to create a Data Model. However, it helps to have both. To get a sense of this, click the Manage Data Model button on the Data ribbon. Assuming Power Pivot isn’t checked in the COM Add-ins list, you will see the dialog box in Figure 4.19. If you click Enable, Power Pivot is automatically checked in the COM Add-ins list, and the Power Pivot window opens so that you can see and manage the data in the Data Model.

09953_ch04_ptg01_132-182.indd 152 04/03/19 12:18 PM

4-3 Data analysis with power pivot 1 5 3

For the rest of this section, we will be using the Power Pivot add-in, so make sure it is loaded by checking its item in the COM Add-ins list. You will know Power Pivot is loaded when there is a Power Pivot tab and associated ribbon in Excel, as shown in Figure 4.20. If you click its Manage button, the Power Pivot window in Figure 4.21 appears. This is where you can see your Data Model, if there is one, and manage it as discussed in this section. You can close the Power Pivot window at any time, and Excel will remain open. Alternatively, you can leave the Power Pivot window open and switch between it and the Excel window in the usual way. (If they are not both visible, click the Excel button, second from left at the top of the Power Pivot window, to go from Power Pivot to Excel. In the other direction, click the Manage button on Excel’s Power Pivot ribbon to go from Excel to Power Pivot.) Note the Save button and the Save options in the File menu in the Power Pivot window. These work exactly like the usual Excel Save options—that is, they save the active workbook, along with its Data Model, if any. This enables you to save from either the Power Pivot window or the Excel window.

It is easy to go back and forth between the Excel window and the Power Pivot window. Both can be open at the same time.

Figure 4.19 Enabling the Data Analysis Add-ins

Figure 4.20 Power Pivot Ribbon

Figure 4.21 Power Pivot Window (with No Data Model Yet)

09953_ch04_ptg01_132-182.indd 153 04/03/19 12:18 PM

1 5 4 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

4-3a Basing Pivot Tables on a Data Model The name Power Pivot suggests that it can be used to create pivot tables, so let’s start there. As you saw in Figure 4.18, you can create a pivot table from a Data Model regard- less of whether Power Pivot is loaded. However, if Power Pivot is loaded, you can click the PivotTable dropdown arrow in the Power Pivot window to see the options in Figure 4.22. You will probably choose the top PivotTable option to get the usual blank pivot table, but you can experiment with the other options. For example, the Two Charts options provide two pivot charts, both based on the Data Model, and you can then populate them in differ- ent and independent ways. So, Power Pivot does indeed provide options for creating pivot tables and pivot charts, but it provides much more.

Figure 4.22 Pivot Table Options in Power Pivot Window

4-3b Calculated Columns, Measures, and the DAX Language The feature that sets Power Pivot apart from regular Excel pivot tables is the relatively new Data Analysis Expressions (DAX) language for performing calculations. It is still pos- sible to create “calculated fields” for regular pivot tables, but these are totally outclassed by the possibilities DAX offers. In fact, in terms of data analysis, learning DAX might be one of the best things you can do to increase your value in the workplace. As one DAX and Power Pivot expert, Rob Collie, states in the forward to his book Power Pivot Alchemy (2014),

“Business Intelligence consulting firms have started to specifically target people who know DAX. As in, if you know Power Pivot, you are their ideal candidate. Doesn’t matter whether you were a Business Intelligence professional yesterday or not – just that you know Power Pivot.”

The relatively new DAX language is used primarily to create calculated columns and measures.

09953_ch04_ptg01_132-182.indd 154 04/03/19 12:18 PM

4-3 Data analysis with power pivot 1 5 5

This section provides an overview of DAX, but it illustrates only a few of the many possibilities. If you want to learn more, you should read a book on DAX such as Rob Collie’s or either of the books listed in the references by Ferrari and Russo. DAX is built around formulas and functions. Some of these are very similar to Excel formulas and functions, but some have no Excel counterparts. There are well over 100 DAX functions.

To get started, take another look at the Power Pivot window in Figure 4.22. This looks like a regular Excel workbook, with rows, columns, and table tabs at the bottom. However, there are important differences. The main difference is that you cannot edit individual cells. The data you see—the data in the Data Model—are linked to a data source and cannot be changed except by changing the source data. The two main things you can do in the Power Pivot window is add calculated columns and add measures, and you do each of these with DAX formulas. A calculated column is basically like a new column in Excel, based on data from other columns in the selected table or even columns in related tables. A measure is a summary function of some type.7 Measures are intended for use in the Values area of pivot tables.

A calculated column is a new column in a table of a Data Model. It is based on fields in the current table or related tables. A measure is a summariza- tion of columns in a Data Model, or a function of other measures, that is intended for use in the Values area of pivot tables. Both are created with DAX formulas.

7 When the Power Pivot add-in was introduced for Excel 2010, the term “measure” was used. Then Microsoft decided to change it to “calculated field” for Excel 2013. Based on user complaints, the term has been changed back to “measure.”

Referring to the data in Figure 4.22, one simple measure, the type you saw in the previous chapter, is “Sum of SalesAmount.” For example, this could be used in a pivot table to break down the sum of sales amount by product and/or order date. However, DAX allows you to create measures that are considerably more complex than simple sums. The following extended example illustrates several possibilities.

EXAMPLE

4.4 USING DAX ON THE CONTOSO DATA Starting with the results in the file Contoso Import Finished.xlsx (from Example 4.1), where seven tables were imported into an Excel Data Model, create some useful calculated columns and measures with DAX and use them to create insightful pivot tables.

Objective To see what DAX calculated columns and measures look like and how they can be used in pivot tables.

Solution For illustration, we will create two types of calculated columns. (You can see the results in the Contoso Pivot Table Finished .xlsx file.) The first is like a typical new column in Excel. We’ll call it Profit, calculated as SalesAmount minus ReturnAmount minus TotalCost. To create it, do the following:

1. Referring to Figure 4.22, click anywhere in the right blank column, the column with the Add Column heading. 2. Type an equals sign (which you’ll see in the formula bar), click the SalesAmount heading, type a minus sign, click the

ReturnAmount heading, type another minus sign, click the TotalCost heading, and press Enter. As you do this, you will

09953_ch04_ptg01_132-182.indd 155 04/03/19 12:18 PM

1 5 6 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

see your DAX formula being created in the formula bar. The final formula should be =Sales[SalesAmount]-Sales[Retur- nAmount]-Sales[TotalCost]. Although this looks like an Excel formula (the type you’d create from an Excel table), it is really a DAX formula. The syntax is that column names are in square brackets, preceded by table names. (The table names can be omitted if there is no ambiguity, but it is good practice to include them.)

3. The default heading for the new column is Calculated Column 1. This is too generic, so right-click the heading, select Rename Column, and type Profit.

Alternatively, you can combine steps 2 and 3 by typing Profit:= Sales[SalesAmount]- Sales[ReturnAmount]-Sales[TotalCost] in the formula bar. You type the name of the calculated column, then colon equals (:=), and then the formula.

When you type the formula and then press Enter, the entire column fills with the profit calculations. This illustrates that you can’t create a partial column with DAX; you must fill an entire column. Also, because there are 33,565 rows in this table, DAX must calculate Profit for each of these rows. This is a critical distinction between a calculated column and a measure.

The second example of a calculated column uses a DAX function, RELATED, that has no counterpart in Excel. It is used to get data into one table from another related table. To use this function, select the Products table, click anywhere in the first blank column, and type ProductCategory:=RELATED( to see the list in Figure 4.23. It shows all columns from all tables. Choose ProductCategoryName from the ProductCategories table, type ), and press Enter. Then repeat, creating a ProductSubcategory calculated column from the ProductSubcategoryName field in the ProductSubcategories table.

With a calculated column, each row’s value in the column must be calculated.

Figure 4.23 Using the RELATED Function

You now have information on product category and product subcategory in the Products table. But why would you want this, given that this information already exists in two related tables? After all, relational database tables are structured specifically to avoid redundancy. There are two reasons. First, you can now hide the ProductCategories and ProductSubcate- gories tables (by right-clicking their sheet tabs and selecting Hide from Client Tools). Then when you use the Data Model to create pivot tables, all information about products will be in a single Products table, which is less confusing to a user.

The RELATED function is used to bring data from a related table into a table.

09953_ch04_ptg01_132-182.indd 156 04/03/19 12:18 PM

4-3 Data analysis with power pivot 1 5 7

The second reason is that you can now build a hierarchy in the Products table. Do this as follows:

1. Switch to diagram view in the Power Pivot window. 2. Move the cursor over the Products table. You will see two buttons at the top right, Create Hier-

archy and Maximize. Click Maximize to expand the table and then click Create Hierarchy. 3. You will see a new Hierarchy field in the list, which you can rename Product Structure.

Then drag the fields ProductCategory, ProductSubcategory, BrandName, and Product- Name, in that order, slightly below the hierarchy name, each slightly below the previous field. (This might take a little practice.) The result should look like Figure 4.24.

A hierarchy allows you to drill down in a pivot table, such as from country to state to city to ZIP code.

Figure 4.24 Creating a Product Structure Hierarchy

4. Click the Maximize button—it’s now a Restore button—to restore the table to its original size.

You will soon see how useful this product hierarchy is for drilling down in pivot tables. DAX offers many more functions for creating calculated columns, such as date functions for manipulating dates. Also,

you can experiment with formulas that look like Excel formulas. For example, you can use IF formulas just like in Excel to create calculated columns. But this is enough to get you started.

We now turn to measures, which are arguably even more important than calculated columns. As stated earlier, a measure is any summarization of the data in the Data Model that can be placed in the Values area of pivot tables. Measures can be created implicitly or explicitly. Given the Contoso Data Model developed so far, suppose you create the usual type of pivot table by dragging the new Profit field to the Values area. This will automatically use the Sum summarization; that is, you will see the sum of Profit broken down by any Row or Column fields. Behind the scenes, however, a measure, Sum of Profit, is created implicitly, and this sum is calculated on the fly, depending on how you break it down. For example, if you drag ProductName to the Rows area of the pivot table, Sum of Profit will be calculated for each product and for all products combined (the grand total). The point here is that you are using measures in pivot tables, even if you aren’t aware of it.

However, measures can also be created explicitly, and the only limit is your imagination and your expertise in DAX. There are two ways you can create a measure explicitly:

1. You can type a DAX formula in the Power Pivot window. You can enter it in any table of the Data Model, but you nor- mally select the table most closely associated with the measure.

2. You can click the Measures dropdown arrow on the Power Pivot ribbon in the Excel window, as shown in Figure 4.25. These two items allow you to create new measures or edit existing measures.

A measure is a summariza- tion of data intended for the Values area of a pivot table. The flexibility of measures created by DAX is the real “power” in Power Pivot.

09953_ch04_ptg01_132-182.indd 157 04/03/19 12:18 PM

1 5 8 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

Let’s create a few measures for the Contoso data. These will not only illustrate the mechanics, but they will show the power of measures:

1. In the Power Pivot window, activate the Sales table, select the left cell just below the data, type Revenue:=SUM(Sales [SalesAmount]), and press Enter. (You will get help as you type this.) As with calculated columns, you name the measure, followed by colon equals (:=), and then the DAX formula. This measure is named Revenue. The formula shows in the formula bar, and the sum of all sales amounts appears in the highlighted cell in Figure 4.26.

Figure 4.25 Measures Dropdown List for Power Pivot

Figure 4.26 Creating Revenue Measure in Power Pivot Window

2. Repeat in the cell below, typing Cost:=SUM(Sales[TotalCost]) to create another measure called Cost. 3. For variety, create another measure called Ratio with the New Measure item in Figure 4.25.

When you click this item, a dialog box appears. Fill it in as shown in Figure 4.27. (The format at the bottom is used for displaying values of the measure in pivot tables.) This measure is associated with the Sales table, its name is Ratio, and its formula, =[Revenue]/[Cost], is based on the existing Revenue and Cost measures (inside required square brackets), not original data columns. That is, you can create measures from other measures. This has an important advantage, as you will see in step 5.

4. Create a pivot table based on the Data Model. The field list under the Sales table will include the original Sales fields plus the three measures you just created. (They have an fx symbol next to them to remind you that they are measures.) For now, drag the Ratio measure to the Values area and the Product Structure hierarchy from the Products table to the Rows area. This illus- trates the value of a hierarchy. It allows you to drill down, such as shown in Figure 4.28. The numbers in this pivot table are calculated exactly as you’d want them to be. For example, for all sales in the VCD & DVD subcategory of the TV and Video category, the sum of all revenues is 214.2% of the sum of all costs. In other words, each ratio is a ratio of sums.

A new measure can be defined in terms of existing measures. If those existing measures change, the new measure updates automatically.

09953_ch04_ptg01_132-182.indd 158 04/03/19 12:18 PM

4-3 Data analysis with power pivot 1 5 9

5. To see the flexibility of measures, suppose your boss now tells you that you calculated the Revenue and Cost measures incorrectly. Revenue should be the sum of SalesAmount minus the sum of ReturnAmount. Also, 5% of all costs are con- sidered overhead costs, and this 5% should not be included in the Cost measure. Fortunately, you do not have to start over. You can simply edit the Revenue and Cost measures in the obvious way, using the Manage Measures item in Figure 4.26. Then because Ratio is the ratio of these measures, it updates automatically, as does the pivot table. Try it. You should see that the ratio for VCD & DVD changes to 222.8%.

6. This step goes beyond the level of the book, but it shows how powerful DAX can be. (You can see the details in the fin- ished version of the file.) We hope it will motivate you to look up and learn the functions involved, especially the complex but powerful CALCULATE function: a. Create the “correct” revenue measure with the formula

Net Revenue:=[Revenue]-SUM(Sales[ReturnAmount])

b. Create the year-to-date net revenue in any year with the formula

Net Revenue YTD:=CALCULATE([Net Revenue],DATESYTD(Dates[DateKey]))

c. Create the year-to-date net revenue for the previous year with the formula

Net Revenue YTD-1:=CALCULATE([Net Revenue YTD],DATEADD(Dates[DateKey],-1,YEAR))

Date-related measures are especially useful, such as for comparing year-to-date revenue this year versus last year.

Figure 4.27 Creating Ratio Measure

1 2 3 4 5 6 7 8 9

10 11 12

A B C ProductCategory ProductSubcategory Ratio

Audio 237.7% Cameras and camcorders Cell phones Computers Music, Movies and Audio Books TV and Video

TV and Video Total VCD & DVD

220.4% 234.4%Grand Total

249.4% 225.1% 231.4% 256.3% 212.1% 217.9% 236.7% 214.2%

Car Video Home Theater System Televisions

Figure 4.28 Pivot Table with Ratio Measure and Product Hierarchy

09953_ch04_ptg01_132-182.indd 159 04/03/19 12:18 PM

1 6 0 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

d. Create the percentage change from one year to the next with the formula

Percent Change YTD YOY:=([Net Revenue YTD]-[Net Revenue YTD-1])/[Net Revenue YTD-1]

Then, using a slicer for CalendarYear, the pivot table in Figure 4.29 is easy to create. It shows, for example, that net reve- nue through October 2008 was $30,743,029, whereas net revenue through October 2009 was $30,309,555, 1.41% lower than in 2008. If you change the slicer year to 2008, the comparison will be between 2007 and 2008—and you will see that the per- centage changes are considerably more negative. Evidently, Contoso tried to correct the downward slide in 2009. In contrast, if you change the slicer year to 2010, column C will be blank because there are no data for 2010 in the database.

FEDCBA

CalendarYear

2007

2008

2009

2010

2005

2006

2011

CalendarMonthLabel January

February

March

April

May

June

July

August

September

October

Novemeber

December

Grand Total

$2,774,510

$5,683,470

$8,127,843

$11,279,806

$14,877,739

$17,774,831

$21,266,356

$24,294,299

$27,522,393

$30,743,029

$34,545,456

$38,398,111

$2,713,037

$5,242,963

$8,109,508

$11,146,191

$14,325,849

$17,732,692

$20,959,554

$24,225,011

$27,141,076

$30,309,555

$33,620,194

$36,733,274

−2.22%

−7.75%

−0.23%

−1.18%

−3.71%

−0.24%

−1.44%

−0.29%

−1.39%

−1.41%

−2.68%

−4.34%

Net Revenue YTD-1 Net Revenue YTD Percent Change YTD YOY

Figure 4.29 Year-to-Date Measures

An important aspect of measures is that they are calculated only when they are required in a pivot table. For example, before the Product Structure hierarchy was added to the pivot table in Figure 4.28, there was just one calculation, the ratio for all sales, that is, the sum of sales amounts for all sales divided by the sum of costs for all sales. Then when the Product Structure hierarchy was added, the Ratio measure had to be calculated only for the members of the hierarchy. If you then added another field such as Continent (from the Geography table) to the pivot table, the Ratio measure would have to be calcu- lated again, but only for combinations of product hierarchy items and continents. Contrast this with the calculated Profit col- umn that had to be calculated over 33,000 times when it was created.

There is one common error made by DAX beginners when they create measures. Suppose you create a measure in the Sales table with the formula Profit:=[SalesAmount]-[Return Amount]-[TotalCost]. You will almost surely wonder why this produces an error. It is not a syntax error. The problem is that SalesAmount, ReturnAmount, and TotalCost are columns in the Sales table, not measures, and a measure must be a summarization. There is no summariz- ing in this formula. In contrast, the formula Total Profit:=Sum[SalesAmount]-Sum[Return Amount]-Sum[TotalCost] works fine.

Now, suppose you drag this Total Profit measure to the Values area of a pivot table. Then you have no choice of how it is summarized; its definition indicates that it is summarized by Sum. If you wanted it to be summarized by Average, you should use the Average function instead of the Sum function in the definition of the measure.

One last simple thing you can do with Power Pivot is hide individual columns (or entire tables). To do so, right-click any column (or table tab) in the Power Pivot window and choose Hide from Client Tools. This is especially useful if you are designing the file for others to use and there are certain columns (or tables) that would never be needed in pivot tables. By hiding them, your users won’t ever see them in the PivotTable Fields list, so they won’t be tempted to use them in pivot tables, and there will be less confusion. As an example, key fields are good candidates for hiding. They serve their purpose in relating tables, and they continue to serve this purpose even when they are hidden, but they are usually meaningless in pivot tables.

A measure must be a summarization. For example, it can’t be a simple addition or subtrac- tion of individual fields.

The values of a measure are calculated only as they are needed in a pivot table.

09953_ch04_ptg01_132-182.indd 160 04/03/19 12:18 PM

4-3 Data analysis with power pivot 1 6 1

This section has presented a brief introduction to calculated columns, measures, and the DAX language. As you will find if you explore these topics further on your own, some of the things you can do with DAX are quite complex, and they take time and practice to master. This is the reason Rob Collie, the author mentioned earlier, states that people with these skills are in great demand. To gain insights, companies need to break down massive amounts of data in all sorts of ways, and, at least in the Microsoft world, DAX is the key to doing this successfully.

The pivot table functionality that has been part of Excel for decades is great, but it has its limits in terms of generating the reports that businesspeople need. With Power Pivot and its DAX language, these limits are expanded significantly. It takes a lot of work to learn and master DAX, but this work is well worth the effort. DAX experts are in high demand.

Fundamental Insight

otherwise. Then create a measure called Total Returns as Sum of Return.

e. Create a pivot table with Total Returns in the Values area, Large Store in the Columns area, and the Store Location hierarchy in the Rows area. Discuss what the numbers mean.

16. The P04_14.xlsx file contains the Contoso Data Model discussed in Example 4.4, but without any calculated columns, measures, or hierarchies. Create a Product Structure hierarchy in the Products table, but when you drag the fields to the hierarchy, drag them in the wrong order, such as ProductSubcategory, then BrandName, then ProductCategory, and then ProductName. Does Power Pivot allow this? What happens?

17. The P04_14.xlsx file contains the Contoso Data Model discussed in Example 4.4, but without any calculated columns, measures, or hierarchies. The question here is whether you can have two different hierarchies in the same table: a. In the Stores table, create a hierarchy called Store

Location 1, using the fields RegionCountryName and StateProvinceName from the Geography table.

b. In the Stores table, try to create a second hierar- chy called Store Location 2, using the fields Conti- nent, RegionCountryName, and CityName. Are you allowed to do so?

c. Create two pivot tables on separate worksheets, one summarizing SalesAmount by the first hierarchy and the other summarizing SalesAmount by the second hierarchy (if it’s allowed).

18. The P04_18.xlsx file contains a single-table Data Model on a company’s sales and a single blank worksheet: a. Create a pivot table on the worksheet, based on the

Data Model, and drag Revenue to the Values area and Date to the Rows area.

b. Create a measure, Total Revenue:=Sum([Revenue]), and add it to Values area of the existing pivot table. Do you get the same results as in part a? Explain.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 14. The P04_14.xlsx file contains the Contoso Data Model

discussed in Example 4.4, but without any calculated columns, measures, or hierarchies: a. In the Sales table, create two calculated columns: Net

Revenue as SalesAmount minus ReturnAmount and Cost without Overhead as 0.95 times TotalCost. Then create a pivot table with ProductCategoryName in the Rows area and the two calculated columns in the Values area, each summarized by Sum.

b. In the Sales table, create two measures: Total Net Reve- nue as Sum of Net Revenue and Total Cost without Over- head as Sum of Cost without Overhead. Then create a pivot table (on a new sheet) with ProductCategoryName in the Rows area and the two measures in the Values area.

c. Do these two pivot tables give the same results? Are there any differences between them? Which requires more calculations? Explain.

15. The P04_14.xlsx file contains the Contoso Data Model discussed in Example 4.4, but without any calculated columns, measures, or hierarchies: a. In the Stores table, create a calculated column named

Large Store, which is 1 for all stores with at least 800 square feet and 0 for others. You can do this with an IF formula just like in Excel.

b. In the Stores table, hide all columns except the calculated column.

c. In the Stores table, create a hierarchy called Store Location, using the fields ContinentName, RegionCoun- tryName, StateProvinceName, and CityName from the Geography table, and then hide the Geography table.

d. In the Sales table, create a calculated column called Return, which is 1 if ReturnAmount is positive and 0

09953_ch04_ptg01_132-182.indd 161 04/03/19 12:18 PM

1 6 2 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

b. With this same pivot table selected, choose Calculated Field from the Fields, Items, & Sets dropdown list on the PivotTable Tools Analyze ribbon. In the resulting dialog box, provide the name Unit Price1 and the formula =Revenue/Units Sold. Click Add and then OK. Explain exactly how the calculation in the Sum of Unit Price1 column is made. Which column is better to report, the Average of Unit Price in part a or the Sum of Unit Price1 calculated here?

c. Go back to the Data sheet. Insert another pivot table, place it on a new worksheet, and check the Add to Data Model option.

d. Open the Power Pivot window. In the rows below the data, create four measures: Unit Price2:=[Revenue]/[Units Sold], Unit Price3:=Sum([Revenue])/Sum([Units Sold]), Unit Price4:=Sum([Revenue]/[Units Sold]), and Unit Price5:=Average([Price per unit]). Two of these should produce errors. Which ones, and why? Then in the pivot table, drag each of the other two to the Values area and drag Store to the Rows area. How do the results com- pare to the results in parts a and b?

22. The P04_19.xlsx file contains a Data Model with four related tables about a company’s sales and a single blank worksheet. This problem illustrates a potential problem with pivot tables when there is a many-to-many relationship, in this case between Products and Orders. (The “fix” to this problem is beyond the level of this book, but the issue is indicated here.) Suppose you are asked to find the number of orders that contain a specific product such as Dog Ear Cyclecomputer. a. The natural way is to create a pivot table with OrderID

from the Orders table in the Values area, summarized by Count, and ProductName from the Products table in the Rows area. When you create this pivot table, you should see that the counts are wrong. They show 944, the total number of orders, for each product.

b. Try instead putting OrderID from the Order_Details table in the Values area of the pivot table, again sum- marized by Count. Provide a convincing argument why these counts are correct. For example, check the records for Dog Ear Cyclecomputer in the Order_Details table.

19. The P04_19.xlsx file contains a Data Model with four related tables about a company’s sales and a single blank worksheet: a. Create two calculated TRUE/FALSE columns, Data

Error:=[ShipDate]<[OrderDate] and Shipped Late:=[ShipDate]>[OrderDate]+3.

b. Create a pivot table based on the Data Model to find the number of data errors and the number of orders that were shipped late.

Level B 20. The P04_18.xlsx file contains a single-table Data Model

on a company’s sales and a single blank worksheet. The goal is to create a pivot table that shows the ratio of average revenue per transaction in the Charleston store to average revenue per transaction in all other stores combined, broken down by month: a. Create the calculated columns RevenueCharles-

ton:=[Revenue]*(Sales[Store]="Charleston") and RevenueOther:=[Revenue]*(Sales[Store]<>"Charleston"). Then create the measure Ratio1:=Average([Reve- nueCharleston])/Average([RevenueOther]) and use it in a pivot table, broken down by month. Explain why you don’t get the desired ratios.

b. Modify the Data Model to get the desired ratios. 21. Before Power Pivot, you could create “calculated

fi e l d s ” f o r p ivo t t a b l e s . T h i s p r o b l e m ex p l o r e s calculated fields versus Power Pivot’s measures for ratios. The P04_21.xlsx file contains a list of transactions for a specific product at various stores for the last several months. Each transaction lists the revenue from the transaction and the number of units of the product sold in the transaction. The ratio of these, also listed, is the unit price at the time the product was sold: a. Create a pivot table but do not add it to a Data Model.

Drag Unit Price to the Values area and Store to the Rows area. Are the values in the pivot table meaning- ful? Does it help if you summarize by Average rather than Sum? Explain.

4-4 Data Visualization with Tableau Public As stated in the introduction, visualizing data with various types of graphs is not new. Analysts have been using standard graphs such as histograms and scatterplots for over a century. However, there has recently been an increased emphasis on visualizing data with imaginative types of graphs, and software has been developed to create these visualiza- tions.8 In this section, we will discuss a popular non-Microsoft software package called Tableau Public. This is the free version of the commercial Tableau product developed by Tableau Software, a company that has specialized in visualization techniques since

8 In the introduction to this chapter, we mentioned Microsoft’s stand-alone software package Power BI. If you search the Web for Power BI, you will find “Microsoft Power BI—Data Visualization Software.” This is one more confirmation of the importance of data visualization in today’s business world.

09953_ch04_ptg01_132-182.indd 162 04/03/19 12:18 PM

4-4 Data Visualization with tableau public 1 6 3

its founding in 2003. You can download Tableau Public—for free—for Windows or Mac from https://public.tableau.com/en-us/s/ (or search for Tableau Public in case this link has changed). We will lead you through an example of using Tableau Public, but we also rec- ommend the Resources link at this website. It has over 20 “how to” videos on using the software.

Remember that Tableau Public is stand-alone software; it is not an Excel add-in. To use Tableau Public, you will typically go through the following steps to create and publish what Tableau calls a “viz” (plural “vizzes”):

1. Connect to external data. For our purposes, this will be data from an Excel workbook, but other data sources are possible.

2. If necessary, manipulate the data into a useful form for visualizations. 3. Create one or more charts based on the data. 4. Optionally, combine charts into a dashboard (a set of related charts) or a story (some-

what like a PowerPoint presentation). 5. Publish your viz to the Web.

This last step means that you can view many visualizations—created by other users or even by yourself—on the Web. From the https://public.tableau.com/en-us/s/ site, click the Gallery tab. This lists the “Viz of the Day” for many days and is probably the best way to see what the software can do. Besides, many of these provide interesting examples of current news stories. For example, the Viz of the Day on the day this section was written was about World Cub 2018 soccer match between Tunisia and England. All these visual- izations are intended to tell a story by visual means. The authors started with data and then probably made a few false starts in Tableau Public before they got the viz that told their story best. You will probably go through this same process in creating your own visual- izations. The mechanics are fairly straightforward, but imagination is required to produce insightful visualizations.

The following extended example takes you through the process of creating a viz from the same baseball salary data you have seen before.

Tableau Public is not an Excel add-in or even a Microsoft product. However, it can import data from Excel worksheets.

EXAMPLE

4.5 USING TABLEAU PUBLIC WITH BASEBALL SALARY DATA The file Baseball Salaries with Team Info2.xlsx contains a sheet for each year from 2012 to 2018 listing each player, his team, his position, and his salary for that year. It also contains a Teams sheet that lists each team, the division and league of the team, and the team’s city, state/province, and country. (One team, the Toronto Blue Jays, is in Canada.) How can Tableau Public be used to create insightful charts and a dashboard of team salaries?

Objective To learn the mechanics of Tableau Public for creating a viz of baseball team salaries.

Solution We will go through the basic steps listed earlier. The first step is to connect to the Excel file. To do this, open Tableau Public. You will see a Connect pane on the left where you can select Microsoft Excel and then navigate to the baseball salaries file. Keep in mind that this only creates a connection to the Excel file. Anything you do to the data in Tableau Public will not be reflected in the Excel source file, which doesn’t change at all.

You will see the Excel sheets listed on the left of the Tableau Public window. We want all the Salary 201x sheets to be joined with the Teams sheet. To do this, select all Salary 201x sheets (select the first, then, while holding the Shift key, select the last) and drag them to the Drag sheets here section on the right. This creates a union of all the Salary 201x sheets, where the data from all years are stacked on top of one another. Then drag the Teams sheet to the right of the Salary 201x union.

09953_ch04_ptg01_132-182.indd 163 04/03/19 12:18 PM

1 6 4 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

This joins them through the Team key fields on all sheets. (Yes, Tableau Public knows about relating tables with primary and foreign keys.) The result appears as in Figure 4.30, where you see a preview of the single combined table of data.

Figure 4.30 Imported Data in Tableau Public

For the second step, manipulating the imported data, there is very little to do in this example. There is a row for each player in each year, exactly what you want. However, there are a couple minor changes you can make:

1. The symbol above each field name indicates the data type, but these can be changed. Specifically, the Year field’s data type is currently numeric (the # sign), but to create line charts, it needs to be a date, so click the # sign and change the data type to Date.

2. Several columns won’t be used for any charts, so they can be hidden. To do so, click the dropdown arrow above any field name and select Hide. Do this for the Name, Sheet, Table Name, and Team (Teams) fields.

Undoing or Redoing

The left and right arrows at the top left of the Tableau Public window let you undo or redo actions. The same keystroke combinations as for Excel, Ctrl+z and Ctrl+y, can also be used.

Tableau tip

Now it’s time to create some charts. You can create the first one by clicking the Sheet 1 tab in Figure 4.30. This sheet starts as shown in Figure 4.31. All fields appear on the left, and these can be dragged to the various areas on the right: Pages, Filters, Marks of different types, Columns and Rows fields, and the Sheet 1 area.

Each “worksheet” in Tableau Public contains some type of chart.

09953_ch04_ptg01_132-182.indd 164 04/03/19 12:18 PM

4-4 Data Visualization with tableau public 1 6 5

Dimensions and Measures

Tableau Public uses terminology that has become common in data analysis: dimensions and measures. Essentially, measures are numeric fields you want to summarize, and dimensions are fields you can break measures down by. In terms of pivot tables, measures are fields you typically drag to the Values area, and dimensions are fields you typically drag to the Rows, Columns, or Filters areas. In the current example, Tableau Public has created a hierarchy (Country, StateProvince, City) for the team location dimension. It has also automatically generated the Latitude and Longitude measures from the location dimension for use in maps.

Tableau tip

Figure 4.31 Blank Tableau Public Worksheet

The concepts of dimensions and measures have become very common, and if you plan to be a data analyst, you will see them a lot. This is because you frequently need to break down numeric variables (measures) by categorical variables (dimensions). In Excel, you typically do this with pivot tables, but this is not exclusively a Microsoft tool. Many other software packages have similar functionality, even though the terminology used may differ.

Fundamental Insight

09953_ch04_ptg01_132-182.indd 165 04/03/19 12:18 PM

1 6 6 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

The first chart will be a map of the United States and lower Canada with a bubble for each team. Each bubble will be sized and colored to indicate the team’s relative total salary. There will be filters for Division and Year. To create this chart, do the following:

1. Drag Country, then StateProvince, then City, then Team to the Detail button. This creates a map, and the Longitude and Latitude measures are automatically added to the Columns and Rows areas.

2. Drag Salary to the Color button. Then click the Color button, reduce the opacity to about 50%, click the Edit Colors button, and choose a color theme you like. (We chose Red, with darker shades of red indicating larger salaries.) You will see a color pane on the right. This isn’t needed, so right-click it and select Hide Card to make it disappear.

3. Drag Salary to the Size button. Then click the Size button and increase or decrease the size if you like. Again, you can hide its card on the right as in step 2.

4. Drag Division to the Filters area. In the resulting dialog box, click the All button to show teams in all divisions. Then click the dropdown arrow for Division in the Filters area and check the Show Filter option. This creates a list of checkboxes in the right pane of the window.

5. Drag Year to the Filters area. You will see two dialog boxes. In the first, select Years; in the second, select All. Then, as in step 4, select the Show Filter option. Finally, click the dropdown arrow at the top of the filter box, select Edit Title, and change the title to Year.

6. Click the Show Me button at the top right. This is always available for displaying the possible types of charts. (Those not possible are grayed out.) For now, click the Show Me button again to hide the list of chart types. By the way, if you start experimenting with the chart types and make a mess of things, you can always “undo” multiple times to return to the original map.

7. Right-click the Sheet 1 tab and rename it Team Salary Map.

The finished version should look like Figure 4.32, where the filters have been applied to show only American League team salaries for 2018. If you hover the cursor over any bubble on the map, you will see information about that team.

Figure 4.32 Map of Team Salaries

09953_ch04_ptg01_132-182.indd 166 04/03/19 12:18 PM

4-4 Data Visualization with tableau public 1 6 7

The next chart will include a series of line charts, one for each team, that show total team salary through the years. To create it, do the following:

1. Click the button to the right of the Team Salary Map tab to get a new blank worksheet. Rename this worksheet Team Sal- aries by Year.

2. Drag Year to the Columns area and Salary to the Rows area. This creates a line chart of the total salary of all teams by year. (You could click the Show Me button to change the chart type if you wanted.)

3. Drag Team to the Detail area. Now you see a line chart for each team. 4. Drag Salary to the Color area and change the color to match the color in the previous chart. (All the charts will eventually

be included in a dashboard, so consistent coloring is recommended.) 5. As in the map chart, create filters for Division and Position. 6. Right-click the Year label above the chart and select the Hide option. Then double-click the Salary label on the vertical

axis to get a dialog box where you can change the axis title from Salary to Total Team Salary.

The finished version should look like Figure 4.33, where filters have been applied to show only National West teams and non-pitchers. In addition, if you click any line, that team’s line will be highlighted, as shown in the figure.

Figure 4.33 Line Charts of Team Salaries

The final chart will be series of column charts, one for each year, showing the average salary for each position. To create it, do the following:

1. Open another new worksheet and rename it Average Salary by Position, Year. 2. Drag Year to the Columns area and then drag Position to the right of Year in the Columns area. 3. Drag Salary to the Rows area. Then right-click the Salary button in the Rows area and change the Measure summarization

to Average.

09953_ch04_ptg01_132-182.indd 167 04/03/19 12:18 PM

1 6 8 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

4. Drag Salary to the Color area, change the color to match the color in the previous two charts, change the Measure summa- rization to Average, and hide the card in the right pane.

5. Create a filter for Position. 6. Right-click the Year/Position label above the chart and select the Hide option.

The finished version should look like Figure 4.33, where a filter has been applied to show everyone except the few (highly paid) designated hitters. One thing becomes evident immediately: It pays to be a first baseman!

Figure 4.34 Average Salary by Position and Year

The final step in this example, before publishing to the Web, is to combine the three charts into a dashboard. To do this, do the following:

1. Click the middle “plus sign” button next to the sheet tabs. This opens a blank dashboard, with the existing charts listed in the left pane. Rename this dashboard sheet MLB Sala- ries 2012–2018.

2. Drag Team Salary Map in this list to the open space in the dashboard. 3. Drag Team Salaries by Year in this list below Team Salary Map. (As you drag, you will

see gray sections where you are allowed to drop.) 4. Drag Average Salary by Position, Year in this list just to the right of Team Salaries by Year. You should now have an arrange-

ment with a large map chart on top and two smaller charts below it. Of course, you could rearrange these if you wanted. 5. In the left pane, there is a Size dropdown arrow. This lets you fit the dashboard to your device: desktop, laptop, or phone.

Use the dropdown arrows to select the Automatic option. 6. The chart titles are not needed, so right-click each of them and either select the Hide option or select Edit Title and change

the title to a single space. (The latter option provides blank space on the chart for the filters in the next step.) 7. The five filters created earlier are listed along the right side of the dashboard. It will take less space to compress them and

float them inside the respective charts. To do this, click the name of any filter, such as Division, to see a dropdown arrow

A dashboard in Tableau Public is a set of related charts.

09953_ch04_ptg01_132-182.indd 168 04/03/19 12:18 PM

4-4 Data Visualization with tableau public 1 6 9

in a gray box. Click it and select Multiple Values (dropdown). Then click it again and select Floating. Now you can drag the filter (by the gray area with two white lines at the top) to a blank space on the appropriate chart. Of course, you can also change or clear the filter if you like. After doing this for all filters and clearing the filters, the dashboard should look approximately like Figure 4.35.

Figure 4.35 Dashboard with Floating Filters

8. When a chart is selected, as the map chart is selected in Figure 4.35, you see options in gray at the top left (or right). Click third button, the filter symbol, to make it white. Then click any team’s bubble on the map. The other two charts automat- ically filter for this team. By holding down the Ctrl key, you can select multiple teams in this way, and the rest will be filtered out. For now, click the filter symbol again to remove the filter.

9. You can also create a “highlight” action. To do so, select Actions from the Dashboard menu at the top of the window, and in the Actions dialog box, select Highlight from the Add Action list. Then fill out the resulting dialog box as shown in Figure 4.36, click OK, and click OK once more to back out. Now select any team’s bubble in the map, and that team’s line chart will be highlighted. This is slightly different from the filter in step 8. With a filter, the other teams’ line charts are completely hidden. With a highlight, they are just grayed out. By the way, if you add the filter in step 8 and this highlight action, the filter action takes precedence.

09953_ch04_ptg01_132-182.indd 169 04/03/19 12:18 PM

1 7 0 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

Figure 4.36 Creating a Highlight Action

Obviously, as with any work of art, you could fine-tune your charts and dashboard forever. At some point, however, you will decide it is good enough to publish to the Web. You can do this as follows:

1. Select Save to Tableau Public (or Save to Tableau Public As) from the File menu. 2. You will be asked to log in to your Tableau Public account. If you haven’t set up an

account yet, you can do so for free. Then you are asked for a name for your viz. The name used for this example is MLB Salaries 2012–2018. (It can be the same as the dashboard name, but it doesn’t have to be the same.)

3. Your viz will open in your default browser, where you can edit it slightly if necessary. Then a button below the viz allows you to share its URL with others (like your instructor) or copy its embed code for use in your own Web pages. For more details, watch the publishing video at https://public.tableau.com/en-us/s/resources.

The final viz for this example is available at https://public.tableau.com/profile/dadm#!/vizhome/MLBSalaries2012-2018/ MLBSalaries2012–2018. It is interactive, so try it out. (Make sure you copy and paste the entire URL.)

If you make changes to your work and you try to close Tableau Public, you will be asked if you want to save your “work- book.” Just be aware that this means saving your changes to the Web, not to a file on your hard drive.

This has been a very brief tour of Tableau Public. To learn more, you can view the videos on Tableau Public’s Resource page or you can experiment on your own. To say the least, Tableau Public is feature-rich, and you might want to try a few options we haven’t illustrated here. For example, you might have noticed the Analytics tab near the top left when creating charts. (See Figure 4.33, for example.) Its options depend on the type of chart, but for the line charts in Figure 4.33, it provides the option of including box-whisker plots on the chart, as shown in Figure 4.37. We will close by letting you decide whether these box-whisker plots add insights to the chart or just clutter it up. With data visualizations, there is bound to be plenty of subjectivity.

When you “save” a Tableau Public project, you are publishing it to the Web. You need an account to do this, but you can get one for free.

09953_ch04_ptg01_132-182.indd 170 04/03/19 12:18 PM

4-4 Data Visualization with tableau public 1 7 1

Figure 4.37 Box-Whisker Plots Added with Analytics Option

Level B 26. Tableau Public can also import table data from pdf

files. The P04_26.pdf file (from the Bureau of Labor Statistics) contains two tables. We are interested only in the first table, which contains monthly values of the Consumer Price Index for all urban consumers (the CPI-U index) from 1913 to mid-2018. Proceed as follows: a. Import the data in Table 1 into Tableau Public.

(When you connect to the pdf source file, ask it to scan for All. You will see several items in the left pane. Experiment to see which union of them you need.) There should be 14 columns: Year, a column for each month, and a Table Name column. Hide this last column.

b. It would be better to have only three columns: Year, Month, and CPI. Tableau Public lets you restructure the data in this way with its “pivot” tool. To use it, highlight all month columns and select Pivot from the dropdown arrow above any month heading. The month columns will be collapsed into two long col- umns, which you can rename Month and CPI.

c. Change the data type of the Year column to Number (whole).

d. Change the data type of CPI to Number (decimal). e. Dates are tricky to work with in Tableau Public. (See

https://onlinehelp.tableau.com/current/pro/desktop/ en-us/functions_functions_date.html for help.) The

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 23. The P04_23.xlsx file contains annual CO2 emissions

data for world countries. (This is basically the same data set used in the training videos on the Tableau Public website.) Use Tableau Public to create at least two charts of the data. Then integrate these charts into a dashboard and publish your results to the Web.

24. The P04_24.xlsx file contains data on over 6000 Airbnb listings in New York City. Use Tableau Public to create at least two charts of the data. Then integrate these charts into a dashboard and publish your results to the Web.

25. The P04_25.xlsx file lists the top finishers in the last five Boston marathons, grouped by age and type. a. The times are listed as 2:33:53, for example, mean-

ing 2 hours, 33 minutes, and 53 seconds, but they are entered and formatted as clock times. Do whatever it takes to express them as minutes. (Hint: A clock time is really a number, the fraction of time from midnight to the next midnight, so simply multiply this by the number of minutes in a day to convert to minutes.)

b. Import the data into Tableau Public. c. Create at least two charts of the data. Then integrate

these charts into a dashboard and publish your results to the Web.

09953_ch04_ptg01_132-182.indd 171 04/03/19 12:18 PM

1 7 2 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

not sure why it is in the Dimensions list.) Then drag Date to the Columns area and CPI to the Rows area and create a Filter for Date on Year. You should see a line chart of CPI by year, but you should then be able to expand the Year(Date) button to Year(Quarter) and then to Year(Month). This is the advantage of having a Date field.

g. Create a dashboard that looks approximately like Figure 4.38, where each chart has its own Year filter.

27. Visit https://www.olympic.org/olympic-results, where you can search for all kinds of Olympics results. Find data of interest to you, import (or type) the data into Excel, use the data to create a viz in Tableau Public, and publish your results to the Web.

goal here is to calculate a date field called Date, which will be used in a line chart. First, change the data type of Month to Date. Second, click the dropdown arrow above Month and select Create Calculated Field to open a dialog box. Enter MonthNumber for the name of the field, enter the formula Month([Month]) in the body—no equals sign is needed—and click OK. For example, this returns 4 for April. Finally, create another calculated field named Date with the formula MakeDate([Year],[MonthNumber],1). (This is like Excel’s Date function, which also takes three integer arguments.)

f. Create a line chart of CPI. To do so, first drag CPI from the Dimensions list to the Measures list. (We’re

Figure 4.38 Dashboard of CPI Time Series Charts

4-5 Data Cleansing When you study statistics in a course, the data sets you analyze have usually been care- fully prepared by the textbook author or your instructor. For that reason, they are usually in good shape—they contain exactly the data you need, there are no missing data, and there are no “bad” entries (caused by typographical errors, for example). Unfortunately, you cannot count on real-world data sets to be so perfect. This is especially the case when you obtain data from external sources such as the Web. There can be all sorts of problems with the data, and it is your responsibility to correct these problems before you do any serious analysis. This initial step, called cleansing data, can be tedious, but it can prevent totally misleading results later on.

In this section we examine one data set that has a number of errors, all of which could very possibly occur in real data sets. We discuss methods for finding the problems and for correcting them. However, you should be aware of two things. First, the errors in this example are only a few of those that could occur. Cleansing data requires careful detective

09953_ch04_ptg01_132-182.indd 172 04/03/19 12:18 PM

4-5 Data Cleansing 1 7 3

work to uncover all possible errors that might be present. Second, once an error is found, it is not always clear how to correct it. A case in point is missing data. For example, some respondents to a questionnaire, when asked for their annual income, might leave this box blank. How should you treat these questionnaires when you perform the eventual data analysis? Should you delete them entirely, should you replace their blank incomes with the average income of all who responded to this question, or should you use a more complex rule to estimate the missing incomes? All three of these options have been suggested by analysts, and all of them have their pros and cons. Perhaps the safest method is to delete any questionnaires with missing data, so that you don’t have to guess at the missing values, but this could mean throwing away a lot of potentially useful data. The point is that some subjectivity and common sense must be used when cleansing data sets.

Again, cleansing data typically involves careful detective work and some common sense. The process is tedious but often necessary. Fortunately, you can use powerful Excel tools to search for suspicious data values and then fix them.

EXAMPLE

4.6 CUSTOMER DATA WITH ERRORS The Data Cleansing.xlsx file has data on 1500 customers of a particular company. A portion of these data appears in Figure 4.39, where many of the rows have been hidden. How much of this data set is usable? How much needs to be cleansed?

Figure 4.39 Data Set with Bad Data

1 2 3 4 5 6 7 8 9

11 12

1498 1499

1501 1500

Customer 1 2 3 4 5 6 7 8

10 11

1497 1498

1500 1499

70 67 25 51 50 39 55 24

33 34 68 44

64 32

0 1 1 1 0 0 1 1

0 0 0 1

0 0

4 0 8 2 2 1 3 0

2 3 5 4

1 6

2080 0

3990 920

1000 550

1400 0

1080 1390 2540 2160

530 2910

910

62900 23300 48700

137600 101400 139700

50900 50500

88300 120300 121100

64000

121400 91000

151400

East West East West East East

South South

West North South North

South South

North

418-18-5649 065-63-3311 059-58-9566 443-13-8685 638-89-7231 202-94-6453

943-85-8301 047-07-5332 638-19-2849 670-57-4549

366-03-5021 166-84-2698

266-29-0308

SSN Birthdate Age Region Cred Card User Income Purchases Amount Spent 539-84-9599 444-05-4079

08/17/73 08/02/47 10/03/48 03/24/60 12/02/43 11/08/74

07/05/65 11/13/64 07/31/30 07/21/54

09/23/34 10/30/66

09/28/67

10/26/44 01/01/32

A B C D E F G H I

Objective To find and fix errors in this company’s data set.

Solution We purposely constructed this data set to have a number of problems, all of which you might encounter in real data sets. We begin with the Social Security Number (SSN). Presumably, all 1500 customers are distinct people, so all 1500 SSNs should be different. How can you tell if they are? One simple way is as follows.

1. Sort on the SSN column. 2. Once the SSNs are sorted, enter the formula 5If(B35B2,1,0) in cell J3 and copy this formula down column J. This

formula checks whether two adjacent SSNs are equal.

09953_ch04_ptg01_132-182.indd 173 04/03/19 12:18 PM

1 7 4 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

3. Enter the formula 5SUM(J3:J501) in cell J2 to see if there are any duplicate SSNs (see Figure 4.40). As you can see, there are two pairs of duplicate SSNs.

Figure 4.41 Dialog Box for Locating Duplicates

Figure 4.40 Checking for Duplicate SSNs

1 2 3 4 5 6 7

681 685 62

787 328 870

63 35 44 23 22 41

0 0 1 0 1 0

1 4 4 3 4 2

2 0 0 0 0 0

530 1750 2020 1330 1940 1010

159700 149300

44000 153000

49800 138900

North North West North West South

001-05-3748 001-43-2336 001-80-6937 002-23-4874 004-10-8303 004-39-9621

A B C D E F G H I J Customer SSN Birthdate Age Region Cred Card User Income Purchases Amount Spent Duplicates

03/24/36 08/21/63 12/27/54 01/31/76 10/19/76 10/13/57

4. To find the duplicates, select the range from cell J3 down and select Find from the Find & Select dropdown menu on the Home ribbon, with the resulting dialog box filled in as shown in Figure 4.41. In particular, make sure the bottom box has Values selected.

5. Click the Find Next button twice to find the offenders. Customers 369 and 618 each have SSN 283-42-4994, and custom- ers 159 and 464 each have SSN 680-00-1375.

At this point, the company should check the SSNs of these four customers, which are hopefully available from another source, and enter them correctly here. (You can then delete column J and sort on column A to bring the data set back to its original form.)

The Birthdate and Age columns present two interesting problems. When the birthdates were entered, they were entered in exactly the form shown (10/26/44, for example). Then the age was calculated by a somewhat complex formula, just as you would calculate your own age.9 Are there any problems? First, sort on Birthdate. You will see that the first 18 customers all have birthdate 05/17/27—quite a coincidence (see Figure 4.42). As you may know, Excel’s dates are stored internally as integers (the number of days since January 1, 1900), which you can see by formatting dates as numbers. So highlight these 18 birthdates and format them with the Number option (zero decimals) to see which number they correspond to. It turns out to be 9999, the code often used for a missing value. Therefore, it is likely that these 18 customers were not born on 05/17/27 after all. Their birthdates were probably missing and simply entered as 9999, which were then formatted as dates. If birthdate is important for further analysis, these 18 customers should probably be deleted from the data set or their birthdates should be changed to blanks if the true values cannot be found.

9 In case you are interested in some of Excel’s date functions, we left the formula for age in cell D2. (We replaced this formula by its values in the rest of column D; otherwise, Excel takes quite a while to recalculate it 1500 times.) This formula uses Excel’s TODAY, YEAR, MONTH, and DAY functions. Check online help to learn more about these functions.

09953_ch04_ptg01_132-182.indd 174 04/03/19 12:18 PM

4-5 Data Cleansing 1 7 5

Figure 4.42 Suspicious Duplicate Birthdates 1

2 3 4 5 6 7 8 9

11 12 13 14 15

17 16

Customer 64

429 463 466 486 494 607 645 661 730 754 782 813

1045 1068 1131 1179 1329 174

71 71 71 71 71 71 71 71

71 71 71 71 71

71 71

0 0 0 0 0 0 1 0

1 0 0 1 0

1 0

1 4 2 4 2 1 3 5 5 1 4 0 2 4 3 3 4 5 2

490 2100

930 1900 1040

480 1540 2480 2450

510 1980

0 1020 1850 1400 1520 1940 2710

960

50500 120300

62300 155400 116500 103700

75900 155300 147900

38200 77300 51600 47500 82400

138500 67800 44800 83900 29900

East South East East West East

South North

West North West East

South

North North

West

SSN Birthdate Age Region Cred Card User Income Purchases Amount Spent 205-84-3572 279-23-7773 619-94-0553 365-18-7407 364-94-9180 085-32-5438 626-04-1182 086-39-4715 212-01-7062 142-06-2339 891-12-9133 183-25-0406 338-58-7652 715-28-2884 110-67-7322 602-63-2343 183-40-5102 678-19-0332 240-78-9827

05/17/27 05/17/27 05/17/27 05/17/27 05/17/27 05/17/27

05/17/27 05/17/27 05/17/27 05/17/27 05/17/27

05/17/27 05/17/27

18 19 20

71 71 69

0 0 0

East West East

05/17/27 05/17/27 01/09/30

05/17/27

05/17/27 05/17/27

A B C D E F G H I

Figure 4.43 Negative Ages: A Y2K Problem 1

2 3 4 5 6 7 8 9

11 12 13

Customer 148 324 426 440

1195 1310 589 824

229 1089 1037

922

–31 –30 –30 –30 –30 –30 –29 –29

–28 –28 –27

–29

0 0 0 1 0 0 1 1

0 0 1

8 2 2 1 4 2 2 2

1 5 3

3960 1000 1100

470 1960

980 1030 1070

450 2580 1510

3000

63800 142500

68400 113600

40600 91800 59300

9999

26700 90000

128300

35400

South North North West West West West North

East South East

East

968-16-0774 618-84-1169 806-70-0226 380-84-2860 776-44-8345 376-25-7809

964-27-4755 808-29-7482 594-47-1955

329-51-3208

SSN Birthdate Age Region Cred Card User Income Purchases Amount Spent 237-88-3817 133-99-5496

09/29/28 10/19/28 10/14/28 10/17/28 04/16/27 11/02/27

01/29/27 02/28/27 08/10/25

03/21/28

08/11/29 05/13/28

A B C D E F G H I

It gets even more interesting if you sort on the Age variable. You will see that the first 12 customers after sorting have neg- ative ages (see Figure 4.43). You have just run into a Y2K (year 2000) problem. These 12 customers were all born before 1930. Excel guesses that any two-digit year from 00 to 29 corresponds to the 21st century, whereas those from 30 to 99 correspond to the 20th century.10 This guess was obviously a bad one for these 12 customers, and you should change their birthdates to the 20th century. An easy way to do this is to highlight these 12 birthdates, select Replace from the Find & Select dropdown list, fill out the resulting dialog box as shown in Figure 4.44, and click the Replace All button. This replaces any year that begins with 202, as in 2028, with a year that begins with 192. (Always be careful with the Replace All option. For example, if you enter /20 and /19 in the “Find what:” and “Replace with:” boxes, you will not only replace the years, but the 20th day of any month will also be replaced by the 19th day.) If you copy the formula for Age that was originally in cell D2 to all of column D, the ages should recalculate automatically as positive numbers.

10 To make matters even worse, a different rule was used in earlier versions of MS Office. There is no guarantee that Microsoft will continue to use this same rule in future editions of Office. However, if you enter four-digit years from now on, as you should, it won’t make any difference.

The Region variable presents a problem that can be very hard to detect—because you usually are not looking for it. There are four regions: North, South, East, and West. If you sort on Region and scroll down, you will find a few East values, a few North values, a few South values, and a few West values, and then the East values start again. Why aren’t the East values all together? If you look closely, you will see that a few of the labels in these cells—those at the top after sorting—begin with a space. The person who entered them inadvertently entered a space before the name. Does this matter? It certainly can. Suppose you create a pivot table, for example, with Region in the Rows area. You will get eight row categories, not four. Therefore, you should delete the extra spaces. The most straightforward way is to use Replace from the Find & Select dropdown menu in the obvious way. (Excel also has a handy TRIM function that removes any leading or trailing spaces from a label.)

A slightly different problem occurs in the Cred Card User column, where 1 corresponds to credit card users and 0 corre- sponds to nonusers. A typical use of these numbers might be to find the proportion of credit card users, which you can find by entering the formula 5AVERAGE(F2:F1501) in some blank cell. This should give the proportion of 1’s, but instead it gives an

09953_ch04_ptg01_132-182.indd 175 04/03/19 12:18 PM

1 7 6 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

error (#DIV/0!). What is wrong? A clue is that the numbers in column F are left-justified, whereas numbers in Excel are usually right-justified. Here is what might have happened. Data on credit card users and nonusers might initially have been entered as the labels Yes and No. Then to convert them to 1 and 0, someone might have entered the formula IF(F45“Yes”,“1”,“0”). The double quotes around 1 and 0 cause them to be interpreted as text, not numbers, and no arithmetic can be done on them. (In addition, text is typically left-justified, the telltale sign seen here.) Fortunately, Excel has a VALUE function that converts text entries that look like numbers to numbers. So you should form a new column that uses this VALUE function on the entries in column F to convert them to numbers. (Specifically, you can create these VALUE formulas in a new column, then do a Copy and Paste Special as Val- ues to replace the formulas by their values, and finally cut and paste these values over the original text in column F.)

Next consider the Income column. If you sort on it, you will see that most incomes are from $20,000 to $160,000. However, there are a few at the top that are much smaller, and there are a few 9999s (see Figure 4.45). By this time, you can guess that the 9999’s correspond to missing values, so unless these true values can be found, these customers should probably be deleted if Income is crucial to the analysis (or their incomes should be changed to blanks). The small numbers at the top take some educated guesswork. Because they range from 22 to 151, a reasonable guess, and hopefully one that can be confirmed, is that the person who entered these incomes thought of them as “thousands” and simply omitted the trailing three zeroes. If this is indeed correct, you can fix them by multiplying each by 1000. (There is an easy way to do this. Enter the multiple 1000 in some blank cell, and press Ctrl+c to copy it. Next, highlight the range G2:G12, click the Paste dropdown menu, select Paste Special, and check the Multiply option. Remember this trick. You can use it often.)

Figure 4.45 Suspicious Incomes 1

2 3 4 5 6 7 8 9

11 12 13 14 15

17 16

Customer 439 593

1343 925

1144 460 407 833 233 51

816 824 518 570 605 796

28 38 56 66 64 27 39 38 67 65 37

�29 24 28 40 45

0 0 1 0 0 0 1 0 0 0 1 1 1 1 1 0

8 5 5 6

9999 3 4 4 6 2 2 2 2 3 0 3

4160 2460 2600 2980 9999 1500 2000 1970 2950 1010

900 1070

940 1520

0 1570

22 25 43 55 71 81 88

104 138 149 151

9999 9999 9999 9999 9999

West East West North North East East West

West North North West South

North North

South

SSN Birthdate Age Region Cred Card User Income Purchases Amount Spent 390-77-9781 744-30-0499 435-02-2521 820-65-4438 211-02-9333 756-41-9393 241-86-3823 908-76-1846 924-59-1581 669-39-4544 884-27-5089 376-25-7809 378-83-7998 758-72-4033 600-05-9780 918-32-8454

06/03/70 05/04/60 08/24/42 11/12/32 08/13/34 05/14/71 07/03/59 09/17/60 05/12/31 10/05/33 03/05/62 11/02/27 11/02/74 11/07/70 07/10/58 03/22/54

A B C D E F G H I

Finally, consider the Purchases (number of separate purchases by a customer) and Amount Spent (total spent on all purchases) columns. First, sort on Purchases. You will see the familiar 9999s at the bottom. In fact, each 9999 for Purchases has a correspond- ing 9999 for Amount Spent. This makes sense. If the number of purchases is unknown, the total amount spent is probably also unknown. You can effectively delete these 9999 rows by inserting a blank row right above them. Excel then automatically senses the boundary of the data. Essentially, a blank row or column imposes a separation from the “active” data.

Now we examine the remaining data for these two variables. Presumably, there is a relationship between these variables, where the amount spent increases with the number of purchases. You can check this with a scatterplot of the (nonmissing) data, which is shown in Figure 4.46. There is a clear upward trend for most of the points, but there are some suspicious outliers at the bottom of the plot. Again, you might take an educated guess. Perhaps the average spent per purchase, rather than the total amount

Figure 4.44 Dialog Box for Correcting the Y2K Problem

09953_ch04_ptg01_132-182.indd 176 04/03/19 12:18 PM

4-5 Data Cleansing 1 7 7

Figure 4.46 Scatterplot with Suspicious Outliers 5000

4500

4000

3500

3000

A m

ou nt

S pe

2500

2000

1500

1000

500

0 0 2 4 6

Purchases

Amount Spent vs Purchases

8 10

Again, cleansing data typically involves careful detective work and some common sense. The bad news is that it is tedious. The good news is that you can use the powerful Excel tools we have discussed to search for suspicious data values and then fix them.

Level B 32. The file P04_32.xlsx contains data imported into

Excel from Microsoft’s Northwind database. There are worksheets for the company’s customers, products, product categories, and transactions. Each transaction is for a product purchased by a customer, but if a customer purchases multiple products at the same time, there are several corresponding rows in the Transactions table, one for each product purchased. The ID columns allow you to look up names of customers, products, and product categories. However, some of the IDs in the Transactions sheet have purposely been corrupted. There can be three reasons. First, an ID in the Transactions sheet might not correspond to any customer, product, or product category. Second, because each order is by a single customer, a given OrderID should correspond to only one CustomerID. Third, a given product ID should always correspond to the same product category ID. Besides the corrupted IDs, there is one other potential type of error, concerning dates. Shipping dates can be blank (for orders that haven’t yet shipped), but they shouldn’t be before the corresponding order dates. Find all corrupted IDs and shipping dates in the Transactions sheet. Highlight all bad data in yellow. You don’t need to change them (because in most cases there is no way of knowing the correct values).

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 28. The file P04_28.xlsx contains a data set that represents

30 responses from a questionnaire concerning the president’s environmental policies. Each observation lists the person’s age, gender, state of residence, number of children, annual salary, and opinion of the president’s environmental policies. Check for bad or suspicious values and change them appropriately.

29. The file P04_29.xlsx contains the following data on movie stars: the name of each star, the star’s gender, domestic gross (average domestic gross of the star’s last few movies), foreign gross (average foreign gross of the star’s last few movies), and income (current amount the star asks for a movie). Check for bad or suspicious val- ues (including names) and change them appropriately.

30. The file P04_30.xlsx contains data on a bank’s employees. Check for bad or suspicious values and change them appropriately.

31. The file P04_31.xlsx contains data on 500 randomly selected households. Check for bad or suspicious values and change them appropriately.

spent, was entered for a few of the customers. This would explain the abnormally small values. (It would also explain why these outliers are all at about the same height in the plot.) If you can locate these outliers in the data set, you can multiply each by the corresponding number of purchases (if your educated guess is correct). How do you locate them in the data set? First, sort on Amount Spent, then sort on Purchases. This will arrange the amounts spent in increasing order for each value of Purchases. Then, using the scatterplot as a guide, scroll through each value of Purchases (starting with 2) and locate the abnormally low values of Amount Spent, which are all together. This procedure is a bit tedious, but it is better than working with invalid data.

09953_ch04_ptg01_132-182.indd 177 04/03/19 12:18 PM

1 7 8 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

4-6 Conclusion The material in the previous two chapters represents the bare minimum people who analyze data should know. Most of these concepts and methods have existed for many years. Even pivot tables, arguably the most important method in these chapters, have existed for over two decades. In contrast, the material in sections 4-2, 4-3, and 4-4 of this chapter is relatively new and represents the foreseeable future of data analysis. The goal is always to gain important, often hidden insights from the massive amount of data now available. Fortunately, as you have seen in this chapter, powerful and user-friendly software such as the “Power” add-ins for Excel and Tableau Public have become available and will continue to be developed. There is little doubt that the people who master this software will become increasingly valuable.

Summary of Key Terms TERM EXPLANATION EXCEL PAGES Business intelligence (BI) An increasingly popular term referring to the

insights gained from data analysis

Self-service BI Refers to business intelligence that can be gener- ated by employees in all areas of business, without need for IT department help, by using powerful, user-friendly software

Data visualization Imaginative graphs for providing insights not immediately obvious from numbers alone

power Query A recent set of Excel tools for importing external data into Excel from a variety of sources

Get & Transform Data group on Data ribbon

Single-table database A database with all data in a single table

Flat file An older name for a single-table database

relational database A database with multiple tables related by key fields, structured to avoid redundancy

primary key A field in a database table that serves as a unique identifier

autonumber key A primary key that automatically assigns consecu- tive integer values

Foreign key A field in a database table that is related to a pri- mary key in another table

One-to-many relationship A relationship between two tables where each record in the “one” table can be associated with many records in the “many” table but where each record in the “many” table can be associated with only one record in the “one” table

Many-to-many relationship A relationship between two tables where each record in each table can be associated with many records in the other table

Linking table A table used to transform a many-to-many relation- ship between two tables into two one-to-many rela- tionships; usually composed mostly of foreign keys

relational database management system (rDBMS)

A complete system, often server-based, for man- aging one or more corporate relational databases (Microsoft SQL Server, Oracle Database, IBM Db2, MySQL, and others)

Query A statement that specifies exactly which data a user requires from a database

Structured Query Language (SQL) A language developed to express queries that can be used (with minor modifications) for all RDBMS

09953_ch04_ptg01_132-182.indd 178 04/03/19 12:18 PM

4-6 Conclusion 1 7 9

C.4. A linking table is required for a many-to-many relation- ship, like between orders and products. Explain what fields a linking table must include at the very least. Give examples of other fields it might also include.

C.5. What is the role of an Excel Data Model? C.6. Can you see data in an Excel Data Model if Power

Pivot is not loaded? Can you use data in an Excel Data Model in a pivot table if Power Pivot is not loaded?

C.7. When you want to filter data for import into Excel, you could filter in the Query Editor, or you could import all the data and then filter inside Excel? What are the pros and cons of these two approaches?

C.8. Suppose you try to create a measure called Profit with the formula =[Revenue]-[Cost], where Revenue and

Conceptual Questions C.1. What does the term “self-service BI” mean, and why is

it important in today’s business world? C.2. Explain how primary and foreign key fields work and

why they are needed in a relational database. C.3. Suppose your company takes orders for its products,

which are supplied to your company by vendors. You want to create a relational database of this information. Discuss the relationship between orders and products and between products and vendors. What tables would you create for this database, and what would you include in these tables?

TERM EXPLANATION EXCEL PAGES

excel Data Model A recent Microsoft technology used to mimic rela- tional databases, but all within Excel

Visible in Power Pivot window

power pivot A recent Excel add-in for creating more powerful pivot table reports than are possible with “regular” pivot tables

Load through COM add-ins list

power pivot window A window for viewing and managing the data in an Excel Data Model

Open with Manage button on Power Pivot ribbon

Data analysis expressions (DaX) language

A language recently developed to create calculated columns and measures in Power Pivot

Calculated column A new column created in the Data Model by using a DAX formula

Power Pivot window

Measure A summarization of data in a Data Model created with a DAX formula and used in the Values area of a pivot table

Power Pivot window

hierarchy A natural sequence of fields such as Country, State, City that can be used to drill down in a pivot table

Power Pivot window

tableau public A free software package developed by Tableau Software for creating data visualizations

Dimensions and Measures General terms used by Tableau Public and others, where measures are numeric fields to be summa- rized and dimensions are usually categorical fields used to break measures down by

Dashboard As used by Tableau Public, a collection of related charts used to tell different aspects of the same basic story

Story As used by Tableau Public, a sequence of visual- izations that tell a data narrative, somewhat like a sequence of PowerPoint slides

Viz Short for visualization, any workbook created in Tableau Public containing charts, dashboards, and/ or stories

Cleansing data Process of removing errors from a data set

Key Terms (continued)

09953_ch04_ptg01_132-182.indd 179 04/03/19 12:18 PM

1 8 0 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

Cost are columns in one of your tables. Why will this produce an error? What should you do to fix it?

C.9. What is the role of a hierarchy in an Excel Data Model? Could you have created something similar in Excel before Data Models and Power Pivot?

C.10. Tableau Public uses the terms “dimensions” and “measures.” What do these correspond to (at least usually) in Excel pivot tables? Why are they so important to data analysis in general?

C.11. What is the “Y2K” problem with dates in Excel (and other software)? What should you do to avoid the problem?

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A Problems 33 to 37 ask you to import data from the Shirt Orders.mdb file into Excel. This is an Access database file with three related tables: Customers, Orders, and Products.

33. Find all records from the Orders table where the order was placed in 2015 or 2016, the product ID is from 1 to 5, the customer ID is not 7, and the number of units ordered is at least 75. Import all fields in the Orders table for each of these records into an Excel table (not a Data Model).

34. Find all records from the Orders table that correspond to orders for between 50 and 100 items (inclusive) made by the customer Rags to Riches for the product Short-sleeve Polo. Import the dates, units ordered, and discounts for each of these orders into an Excel table (not a Data Model).

35. The goal here is to create a table in Excel that contains the order date, customer name, and product description for all orders that satisfy conditions on orders, products, and customers. Proceed as follows: a. Use Power Query to import all three tables into Excel

as tables (not a Data Model). The only use for Query Editor is to remove the columns not used for this problem: Street, City, State, Zip, Phone for customers, Discount for orders, and UnitPrice for products.

b. Add three new fields, Customer, Product, and Gender, to the Orders table in Excel and use VLOOKUP func- tions to fill them.

c. Filter the Orders table as necessary. For this problem, use a “>75” filter for units ordered, “Both” for gender, and both “Threads” and “Shirts R Us” for customer.

36. Solve the previous problem a different way. Use Power Query to import the same data but load the data into a Data Model. Then use the RELATED function to bring the customer, product, and gender information into the Orders table and finally apply the same filters as in the previous problem.

37. The company would like to know how many units of its products designed for each gender category (men,

women, and both genders) were sold to each customer during each quarter of the past five years (Q1 of 2012 through Q4 of 2016). Use Power Query to import the appropriate data into an Excel pivot table report (not a Data Model or a table) and then manipulate the pivot table to show the required information.

Problems 38 to 42 ask you to import and then analyze data from the Foodmart.mdb file. This is an Access database file, another “practice” Microsoft database, with five related tables: Customers, Dates, Facts, Products, and Stores. The Facts table, with over 250,000 rows, has a row for each sale of a product during part of 1997 and 1998. Four of its fields are foreign keys relating the sale to the other four tables. Its other two fields, the “facts,” are Revenue and UnitsSold from the sale.

38. Use Power Query to create a Data Model consisting of relevant fields from the Facts and Products tables, where “relevant” means they are needed for the following. Use a pivot table to find the top three products in terms of the highest number of sales transactions.

39. Do the same as in the previous problem but now include the Dates table in the import and find the top four products in terms of the highest number of sales transactions in 1998.

40. Use Power Query to create a Data Model consisting of relevant fields from the Facts, Products, and Customers tables. The latter should include at least the Country and City fields. In the Power Pivot window, create a hierarchy called Country-City consisting of the Country and City fields. Then create a pivot table based on the Data Model to find the three cities that have generated the most revenue in the Beverages product department in their respective countries.

41. Use Power Query to create a Data Model consisting of relevant fields from the Facts, Products, Stores, and Dates tables. Then create two hierarchies, Product Structure and Store Location. The first should contain the fields ProductFamily, ProductDepartment, ProductCategory, and ProductSubcategory. The second should contain the fields Country, Region, StateProvince, and City. Use this to create a pivot table and accompanying pivot chart that breaks down total revenue by the two hierarchies, the first in the Rows area and the second in the Columns area. Also, include a slicer for the day of the week (Monday, Tuesday, etc.). Do whatever it takes to produce a pivot chart like in Figure 4.47, which shows total revenue on the weekends.

42. Use Power Query to create a Data Model consisting of relevant fields from the Facts and Products tables. Because you don’t plan to use the various “ID” key fields in any pivot table, use the Query Editor to remove them. Then create a pivot table of total revenue for each product family. What can you say about this pivot table, and why? What could you do instead to make sure that the ID key fields are never part of any pivot table?

43. The P04_43.txt file is a text file that contains yearly data for the number of licensed drivers (those under 18, those over 85, and total) by gender and state. Use Power

09953_ch04_ptg01_132-182.indd 180 04/03/19 12:18 PM

4-6 Conclusion 1 8 1

Figure 4.47 Total Revenue on Weekends

$0.00 $5,000.00

$10,000.00 $15,000.00 $20,000.00 $25,000.00 $30,000.00 $35,000.00 $40,000.00 $45,000.00 $50,000.00

Frozen Foods Meat Produce Seafood

Food

Product Structure

Store Location

USA - South West

USA - North West

USA - Central West

Sum of Revenue

+ –

Query to import the data into Excel and then save the Excel file. Use appropriate text functions (unless you want to do it manually) to shorten the variable names to names like Arizona Females Young, Arizona Females Old, and Arizona Females All. Explain briefly what the presence of consecutive commas in the text file indicates. How are these imported into Excel?

44. The P04_44.txt file is a text file that lists daily values of an air quality index for Los Angeles. The first value in each row is a date such as 20070322. This means March 22, 2007. Use Power Query to import the data into Excel and then save the Excel file. Use appropriate Excel functions to transform the text values such as 20070322 into dates and format them as “mm/dd/yyyy.” Then use Tableau Public to create a line chart of the air quality index by month and publish your results to the Web.

Level B 45. The srn_pcp.txt file is a text file that contains monthly

precipitation from 1895 through January 2005. The srn_data.txt is a “data dictionary” that explains the precipitation file. (The data dictionary mentions two other files, srn_tmp.txt and srn_pdsi.txt. These files are available but they aren’t used here.) Proceed as follows. (Note: You could import the text data directly into Tableau Public, avoiding Excel altogether. However, we find it easier to manipulate the data in Excel than in Tableau Public—except for the pivot tool in step i.) a. Use Power Query to import the precipitation data into

an Excel table (not a Data Model). Use the Query Edi- tor for only one purpose, to change the data type for column 1 to text. (You can do this with the Data Type dropdown on the Transform ribbon.)

b. In Excel, change the column headings for columns 2 to 13 to month names: Jan, Feb, etc. Also, delete the last column, which is empty.

d. Scan down to any year 2005 row. Only January’s pre- cipitation is listed. The rest are missing, denoted by the code -99.9. Perform a search and replace for -99.9, replacing each with a blank.

e. The codes in the Code column are 001 to 048 for the contiguous 48 states. The other codes, above 100, are for regions. Delete all rows with these region codes.

f. The State Codes.xlsx file contains a table of codes and the corresponding states. Copy its data some- where to the right of the imported precipitation data. Then insert a new column, State, to the right of the precipitation Code column and use a VLOOKUP function to fill it with the state names. Copy the State column and paste it over itself as values. Then the columns you copied from the State Codes file are no longer needed, so you can delete them. You can also delete the Code column.

g. Save this Excel file as Precip Data.xlsx. It should have 14 columns: State, Year, and a column for each month.

h. Open Tableau Public and import the data from the Excel file you just created.

i. The data aren’t structured well for analysis. It would be better to have four columns: State, Year, Month, and Precip. Tableau Public has a “pivot” tool for restructur- ing. Highlight all month columns and select Pivot from the dropdown arrow above any month’s heading. This collapses the month columns into two long columns, which you can rename Month and Precip.

09953_ch04_ptg01_132-182.indd 181 04/03/19 12:18 PM

1 8 2 C h a p t e r 4 B u s i n e s s I n t e l l i g e n c e ( B I ) t o o l s f o r D a t a a n a l y s i s

are sized and colored according to total precipitation in any selected year(s), and the line chart shows total pre- cipitation by month for any selected year(s) and state(s).

j. Now that the data are in the proper form, create two charts and a dashboard from them that should look approximately like Figure 4.48. The bubbles in the map

Figure 4.48 Finished Precipitation Dashboard

09953_ch04_ptg01_132-182.indd 182 04/03/19 12:18 PM

CHAPTER 5 Probability and Probability Distributions

CHAPTER 6 Decision Making Under Uncertainty

P A R T 2 PROBABILITY AND DECISION

MAKING UNDER UNCERTAINTY

09953_ch05_ptg01_183-241.indd 183 04/03/19 12:25 PM

GAME AT MCDONALD’S Several years ago, McDonald’s ran a campaign in which it gave game cards to its customers. These game cards made it possible for customers to win hamburgers, french fries, soft drinks, and other fast-food items, as well as cash prizes. Each card had 10 covered spots that could be uncovered by rubbing them with a coin. Beneath three of these spots were “zaps.” Beneath the other seven spots were names of prizes, two of which were identical. For example, one card might have two pictures of a hamburger, one picture of a Coke, one of french fries, one of a milk shake, one of $5, one of $1000, and three zaps. For this card the customer could win a ham- burger. To win on any card, the customer had to uncover the two matching spots (which showed the potential prize for that

card) before uncovering a zap; any card with a zap uncovered was automatically void. Assuming that the two matches and the three zaps were arranged randomly on the cards, what is the probability of a customer winning?

We label the two matching spots M1 and M2, and the three zaps Z1, Z2, and Z3. Then the probability of winning is the probability of uncovering M1 and M2 before uncovering Z1, Z2, or Z3. In this case the relevant set of outcomes is the set of all orderings of M1, M2, Z1, Z2, and Z3, shown in the order they are uncovered. As far as the outcome of the game is concerned, the other five spots on the card are irrelevant. Thus, an outcome such as M2, M1, Z3, Z1, Z2 is a winner, whereas M2, Z2, Z1, M1, Z3 is a loser. Actually, the first of these would be declared a winner as soon as M1 was uncovered, and the second would be declared a loser as soon as Z2 was uncovered. However, we show the whole sequence of M’s and Z’s so that we can count outcomes correctly. We then find the probability of winning using an equally likely argument. Specifically, we divide the number of outcomes that are winners by the total number of outcomes. It can be shown that the number of out- comes that are winners is 12, whereas the total number of outcomes is 120. Therefore, the probability of a winner is 12/120 = 0.1.

This calculation, which shows that, on average, 1 out of 10 cards could be win- ners was obviously important for McDonald’s. Actually, this provides only an upper bound on the fraction of cards where a prize was awarded. Many customers threw their cards away without playing the game, and even some of the winners neglected to claim their prizes. So, for example, McDonald’s knew that if they made 50,000 cards where a milk shake was the winning prize, somewhat less than 5000 milk shakes would be given away. Knowing approximately what their expected “losses” would be from win- ning cards, McDonald’s was able to design the game (how many cards of each type to print) so that the expected extra revenue (from customers attracted to the game) would cover the expected losses.

CHAPTER 5 Probability and Probability Distributions

S o rb

is /S

h u tt e rs

to c k.

c o m

09953_ch05_ptg01_183-241.indd 184 04/03/19 12:25 PM

5-1 Introduction 1 8 5

5-1 Introduction A key aspect of solving real business problems is dealing appropriately with uncertainty. This involves recognizing explicitly that uncertainty exists and using quantitative methods to model uncertainty. If you want to develop realistic business models, you cannot simply act as if uncertainty doesn’t exist. For example, if you don’t know next month’s demand, you shouldn’t build a model that assumes next month’s demand is a sure 1500 units. This is only wishful thinking. You should instead incorporate demand uncertainty explicitly into your model. To do this, you need to know how to deal quantitatively with uncertainty. This involves probability and probability distributions. We introduce these topics in this chapter and then use them in a number of later chapters.

There are many sources of uncertainty. Demands for products are uncertain, times between arrivals to a supermarket are uncertain, stock price returns are uncertain, changes in interest rates are uncertain, and so on. In many situations, the uncertain quantity— demand, time between arrivals, stock price return, change in interest rate—is a numerical quantity. In the language of probability, it is called a random variable. More formally, a random variable associates a numerical value with each possible random outcome.

Associated with each random variable is a probability distribution that lists all of the possible values of the random variable and their corresponding probabilities. A proba- bility distribution provides very useful information. It not only indicates the possible val- ues of the random variable but it also indicates how likely they are. For example, it is useful to know that the possible demands for a product are, say, 100, 200, 300, and 400, but it is even more useful to know that the probabilities of these four values are, say, 0.1, 0.2, 0.4, and 0.3. This implies, for example, that there is a 70% chance that demand will be at least 300.

It is often useful to summarize the information from a probability distribution with numerical summary measures. These include the mean, variance, and standard deviation. As their names imply, these summary measures are much like the corresponding summary measures in Chapters 2 and 3. However, they are not identical. The summary measures in this chapter are based on probability distributions, not an observed data set. We will use numerical examples to explain the difference between the two—and how they are related.

The purpose of this chapter is to explain the basic concepts and tools necessary to work with probability distributions and their summary measures. The chapter then dis- cusses several important probability distributions, particularly the normal distribution and the binomial distribution, in some detail.

Modeling uncertainty, as we will be doing in the next chapter and later in Chapters 15 and 16, is sometimes difficult, depending on the complexity of the model, and it is easy to get so caught up in the details that you lose sight of the big picture. For this rea- son, the flow chart in Figure 5.1 is useful. (A colored version of this chart is available in the file Modeling Uncertainty Flow Chart.xlsx.) Take a close look at the middle row of this chart. You begin with inputs, some of which are uncertain quantities, you use Excel® formulas to incorporate the logic of the model, and the result is probability distributions of important outputs. Finally, you use this information to make decisions. (The abbreviation EMV stands for expected monetary value. It is discussed extensively in Chapter 6.) The other boxes in the chart deal with implementation issues, particularly with the software you can use to perform the analysis. Study this chart carefully and return to it as you pro- ceed through the next few chapters and Chapters 15 and 16.

Before proceeding, we discuss two terms you often hear in the business world: uncertainty and risk. They are sometimes used interchangeably, but they are not really the same. You typically have no control over uncertainty; it is something that simply exists. A good example is the uncertainty in exchange rates. You cannot be sure what the exchange rate between the U.S. dollar and the euro will be a year from now. All you can try to do is measure this uncertainty with a probability distribution.

09953_ch05_ptg01_183-241.indd 185 04/03/19 12:25 PM

1 8 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

In contrast, risk depends on your position. Even though you don’t know what the exchange rate will be, it makes no difference to you—there is no risk to you—if you have no European investments, you aren’t planning a trip to Europe, and you don’t have to buy or sell anything in Europe. You have risk only when you stand to gain or lose money depending on the eventual exchange rate. Of course, the type of risk you face depends on your position. If you are holding euros in a money market account, you are hoping that euros gain value relative to the dollar. But if you are planning a European vacation, you are hoping that euros lose value relative to the dollar.

Uncertainty and risk are inherent in many of the examples in this book. By learning about probability, you will learn how to measure uncertainty, and you will also learn how to measure the risks involved in various decisions. One important topic you will not learn much about is risk mitigation by various types of hedging. For example, if you know you have to purchase a large quantity of some product from Europe a year from now, you face the risk that the value of the euro could increase dramatically, thus cost- ing you a lot of money. Fortunately, there are ways to hedge this risk, so that if the euro does increase relative to the dollar, your hedge minimizes your losses. Hedging risk is an extremely important topic, and it is practiced daily in the real world, but it is beyond the scope of this book.

5-2 Probability Essentials We begin with a brief discussion of probability. The concept of probability is one that you all encounter in everyday life. When a weather forecaster states that the chance of rain is 70%, he or she is making a probability statement. When a sports commentator states that the odds against the Golden State Warriors winning the NBA Championship are 3 to 1, he or she is also making a probability statement. The concept of probability is quite intuitive. However, the rules of probability are not always as intuitive or easy to master. We examine the most important of these rules in this section.

Assess probability distributions of uncertain inputs:

Decide which inputs are important for the model.

1. Which are known with certainty? 2. Which are uncertain?

Two fundamental approaches:

1. Build an exact probability model that incorporates the rules of probability. (Pros: It is exact and amenable to sensitivity analysis. Cons: It is often difficult mathematically, maybe not even possible.)

2. Build a simulation model. (Pros: It is typically much easier, especially with add-ins like DADM_Tools or @RISK, and extremely versatile. Cons: It is only approximate and runs can be time consuming for complex models).

For simulation models, random values for uncertain inputs are necessary.

1. They can sometimes be generated with built-in Excel functions. This often involves tricks and can be obscure.

2. Add-ins like DADM_Tools or @RISK provide functions that make it much easier.

Examine important outputs.

The result of these formulas should be probability distribution(s) of important output(s). Summarize these probability distributions with (1) histograms (risk profiles), (2) means and standard deviations, (3) percentiles, (4) possibly others.

Model the problem.

Use Excel formulas to relate inputs to important outputs, i.e., enter the business logic.

Make decisions based on this information.

Criterion is usually EMV, but it could be something else, e.g., minimize the probability of losing money.

1. If a lot of historical data is available, find the distribution that best fits the data.

2. Choose a probability distribution (normal? triangular?) that seems reasonable. Add-ins like DADM_Tools or @RISK are helpful for exploring distributions.

3. Gather relevant information, ask experts, and do the best you can.

This is an overview of spreadsheet modeling with uncertainty. The main process is in red. The blue boxes deal with implementation issues.

For simulation models, this can be done “manually” with data tables and built-in functions like AVERAGE, STDEV.S, etc. But add-ins like DADM_Tools or @RISK take care of these bookkeeping details automatically.

Use decision trees, made easier with add-in like DADM_Tools or PrecisionTree, if the number of possible decisions and the number of possible outcomes are not too large.

Figure 5.1 Flow Chart for Modeling Uncertainty

09953_ch05_ptg01_183-241.indd 186 04/03/19 12:25 PM

5-2 Probability Essentials 1 8 7

As the examples in the preceding paragraph illustrate, probabilities are sometimes expressed as percentages or odds. However, these can easily be converted to probabilities on a 0-to-1 scale. If the chance of rain is 70%, then the probability of rain is 0.7. Similarly, if the odds against the Warriors winning are 3 to 1, then the probability of the Warriors winning is 1/4 (or 0.25).

There are only a few probability rules you need to know, and they are discussed in the next few subsections. Surprisingly, these are the only rules you need to know. Probability is not an easy topic, and a more thorough discussion of it would lead to considerable math- ematical complexity, well beyond the level of this book. However, it is all based on the few relatively simple rules discussed next.

5-2a Rule of Complements The simplest probability rule involves the complement of an event. If A is any event, then the complement of A, denoted by A (or in some books by Ac), is the event that A does not occur. For example, if A is the event that the Dow Jones Industrial Average will finish the year at or above the 25,000 mark, then the complement of A is that the Dow will finish the year below 25,000.

If the probability of A is P(A), then the probability of its complement, P(A), is given by Equation (5.1). Equivalently, the probability of an event and the probability of its com- plement sum to 1. For example, if you believe that the probability of the Dow finishing at or above 25,000 is 0.25, then the probability that it will finish the year below 25,000 is 1 − 0.25 = 0.75.

A probability is a number between 0 and 1 that measures the likelihood that some event will occur. An event with probability 0 cannot occur, whereas an event with probability 1 is certain to occur. An event with probability greater than 0 and less than 1 involves uncertainty. The closer its probability is to 1, the more likely it is to occur.

Rule of Complements

P1A2 5 1 2 P1A2 (5.1)

5-2b Addition Rule Events are mutually exclusive if at most one of them can occur. That is, if one of them occurs, then none of the others can occur. For example, consider the following three events involving a company’s annual revenue for the coming year: (1) revenue is less than $1 mil- lion, (2) revenue is at least $1 million but less than $2 million, and (3) revenue is at least $2 million. Clearly, only one of these events can occur. Therefore, they are mutually exclu- sive. They are also exhaustive events, which means that they exhaust all possibilities— one of these three events must occur. Let A1 through An be any n events. Then the addition rule of probability involves the probability that at least one of these events will occur. In general, this probability is quite complex, but it simplifies considerably when the events are mutually exclusive. In this case the probability that at least one of the events will occur is the sum of their individual probabilities, as shown in Equation (5.2). Of course, when the events are mutually exclusive, “at least one” is equivalent to “exactly one.” In addition,

09953_ch05_ptg01_183-241.indd 187 04/03/19 12:25 PM

1 8 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

In a typical application, the events A1 through An are chosen to partition the set of all possible outcomes into a number of mutually exclusive events. For example, in terms of a company’s annual revenue, define A1 as “revenue is less than $1 million,” A2 as “revenue is at least $1 million but less than $2 million,” and A3 as “revenue is at least $2 million.” Then these three events are mutually exclusive and exhaustive. Therefore, their probabilities must sum to 1. Suppose these probabilities are P(A1) = 0.5, P(A2) = 0.3, and P(A3) = 0.2. (Note that these probabilities do sum to 1.) Then the additive rule enables you to calculate other probabilities. For example, the event that revenue is at least $1 million is the event that either A2 or A3 occurs. From the addition rule, its prob- ability is

P1revenue is at least $1 million2 5 P1A22 1 P1A32 5 0.5 Similarly,

P1revenue is less than $2 million2 5 P1A12 1 P1A22 5 0.8 and

P1revenue is less than $1 million or at least $2 million2 5 P1A12 1 P1A32 5 0.7 Again, the addition rule works only for mutually exclusive events. If the events over-

lap, the situation is more complex. For example, suppose you are dealt a bridge hand (13 cards from a 52-card deck). Let H, D, C, and S, respectively, be the events that you get at least 5 hearts, at least 5 diamonds, at least 5 clubs, and at least 5 spades. What is the probability that at least one of these four events occurs? It is not the sum of their individual probabilities because they are not mutually exclusive. For example, you could get a hand with 5 hearts and 5 spades. Probabilities such as this one are actually quite difficult to calculate, and we will not pursue them here. Just be aware that the addition rule does not apply unless the events are mutually exclusive.

5-2c Conditional Probability and the Multiplication Rule Probabilities are always assessed relative to the information currently available. As new information becomes available, probabilities can change. For example, if you read that Steph Curry suffered a season-ending injury, your assessment of the probability that the Warriors will win the NBA Championship would obviously change. (It would probably become 0!) A formal way to revise probabilities on the basis of new information is to use conditional probabilities.

Let A and B be any events with probabilities P(A) and P(B). Typically, the probability P(A) is assessed without knowledge of whether B occurs. However, if you are told that B has occurred, then the probability of A might change. The new probability of A is called the conditional probability of A given B, and it is denoted by P(A1B). Note that there is still uncertainty involving the event to the left of the vertical bar in this notation; you do not know whether it will occur. However, there is no uncertainty involving the event to the right of the vertical bar; you know that it has occurred. The conditional probability can be calculated with the following formula.

Addition Rule for Mutually Exclusive Events

P1at least one of A1 through An2 5 P1A12 1 P1A22 1 c 1 P1An2 (5.2)

if the events A1 through An are exhaustive, then the probability is 1 because one of the events is certain to occur.

09953_ch05_ptg01_183-241.indd 188 04/03/19 12:25 PM

5-2 Probability Essentials 1 8 9

The numerator in this formula is the probability that both A and B occur. This proba- bility must be known to find P1AuB2. However, in some applications P1AuB2 and P(B) are known. Then you can multiply both sides of Equation (5.3) by P(B) to obtain the follow- ing multiplication rule for P(A and B).

Multiplication Rule

P1A and B2 5 P1AuB2 P1B2 (5.4)

The conditional probability formula and the multiplication rule are both valid; in fact, they are equivalent. The one you use depends on which probabilities you know and which you want to calculate, as illustrated in Example 5.1.

EXAMPLE

5.1 ASSESSING UNCERTAINTY AT BENDER COMPANY Bender Company supplies contractors with materials for the construction of houses. The company currently has a contract with one of its customers to fill an order by the end of July. However, there is some uncertainty about whether this deadline can be met, due to uncertainty about whether Bender will receive the materials it needs from one of its suppliers by the middle of July. Right now it is July 1. How can the uncertainty in this situation be assessed?

Objective To apply probability rules to calculate the probability that Bender will meet its end-of-July deadline, given the information the company has at the beginning of July.

Solution Let A be the event that Bender meets its end-of-July deadline, and let B be the event that Bender receives the materials from its supplier by the middle of July. The probabilities Bender is best able to assess on July 1 are probably P(B) and P1AuB2. At the beginning of July, Bender might estimate that the chances of getting the materials on time from its supplier are 2 out of 3, so that P(B) = 2/3. Also, thinking ahead, Bender estimates that if it receives the required materials on time, the chances of meeting the end-of-July deadline are 3 out of 4. This is a conditional probability statement that P1AuB2 5 3>4. Then the multiplication rule implies that

P1A and B2 5 P1AuB2P1B2 5 13>4212>32 5 0.5 That is, there is a fifty–fifty chance that Bender will get its materials on time and meet its end-of-July deadline.

This uncertain situation is depicted graphically in the form of a probability tree in Figure 5.2. Note that Bender ini- tially faces (at the leftmost branch of the tree) the uncertainty of whether event B or its complement will occur. Regardless of whether event B occurs, Bender must next confront the uncertainty regarding event A. This uncertainty is reflected in the set of two pairs of branches in the right half of the tree. Hence, there are four mutually exclusive outcomes regarding the two uncer- tain events, as listed to the right of the tree. Initially, Bender is interested in the first possible outcome, the joint occurrence of

Conditional Probability

P1AuB2 5 P1A and B2 P1B2 (5.3)

09953_ch05_ptg01_183-241.indd 189 04/03/19 12:25 PM

1 9 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

events A and B, the top probability in the figure. Another way to compute this probability is to multiply the probabilities asso- ciated with the branches leading to this outcome, that is, the probability of B times the probability of A given B. As the figure indicates, this is 13/4212/32, or 0.5.

P(A and B) = (3/4)(2/3)

P(A and B) = (1/4)(2/3)

P(A and B) = (1/5)(1/3)

P(A and B) = (4/5)(1/3)

Figure 5.2 Probability Tree for Bender Example

5-2d Probabilistic Independence A concept that is closely tied to conditional probability is probabilistic independence. You just saw that the probability of an event A can depend on whether another event B has occurred. Typically, the probabilities P(A), P1AuB2, and P1AuB2 are all different, as in Example 5.1. However, there are situations where all of these probabilities are equal. In this case, A and B are probabilistically independent events. This does not mean they are mutually exclusive. Rather, probabilistic independence means that knowledge of one event is of no value when assessing the probability of the other.

There are several other probabilities of interest in this example. First, let B be the complement of B; it is the event that the materials from the supplier do not arrive on time. We know that P1B2 5 1 2 P1B2 5 1>3 from the rule of complements. How- ever, we do not yet know the conditional probability P1AuB2, the probability that Bender will meet its end-of-July deadline, given that it does not receive the materials from the supplier on time. In particular, P1AuB2 is not equal to 1 2 P1AuB2. (Can you see why?) Suppose Bender estimates that the chances of meeting the end-of-July deadline are 1 out of 5 if the materials do not arrive on time, so that P1AuB2 5 1>5. Then a second use of the multiplication rule gives

P1A and B2 5 P1AuB2P1B2 5 11>5211>32 5 0.0667 In words, there is only 1 chance out of 15 that the materials will not arrive on time and Bender will meet its end-of-July dead- line. This is the third (from the top) probability listed at the right of the tree.

The bottom line for Bender is whether it will meet its end-of-July deadline. After mid-July, this probability is either P1AuB2 5 3>4 or P1AuB2 5 1>5 because by this time, Bender will know whether the materials arrived on time. But on July 1, the relevant probability is P(A)—there is still uncertainty about whether B or B will occur. Fortunately, you can calculate P(A) from the probabilities you already know. The logic is that A consists of the two mutually exclusive events (A and B) and 1A and B2. That is, if A is to occur, it must occur with B or with B. Therefore, the addition rule for mutually exclusive events implies that

P1A2 5 P1A and B2 1 P1A and B2 5 1>2 1 1>15 5 17>30 5 0.5667 The chances are 17 out of 30 that Bender will meet its end-of-July deadline, given the information it has at the beginning of July.

09953_ch05_ptg01_183-241.indd 190 04/03/19 12:25 PM

5-2 Probability Essentials 1 9 1

How can you tell whether events are probabilistically independent? Unfortunately, this issue usually cannot be settled with mathematical arguments. Typically, you need actual data to decide whether independence is reasonable. As a simple example, let A be the event that a family’s first child is male, and let B be the event that its second child is male. Are A and B independent? You could argue that they aren’t independent if you believe, say, that a boy is more likely to be followed by another boy than by a girl. You could argue that they are independent if you believe the chances of the second child being a boy are the same, regardless of the gender of the first child. (Note that neither argument requires that boys and girls are equally likely.)

In any case, the only way to settle the argument is to observe many families with at least two children. If you observe, say, that 55% of all families with first child male also have the second child male, and only 45% of all families with first child male have the sec- ond child female, then you can make a good case that A and B are not independent.

The concept of independence carries over to random variables. Two random variables, X and Y, are independent if any two events, one involving only X and the other involving only Y, are independent. The idea is that knowledge of X is of no help in predicting Y, and vice versa. For example, if X is the amount of rain in Seattle in March and Y is the amount of rain in Seattle in June, it might be realistic to assume that X and Y are independent random variables. March weather probably doesn’t have much effect on June weather. On the other hand, if X and Y are the changes in stock prices of two companies in the same industry from one day to the next, it might not be realistic to assume that X and Y are inde- pendent. The reason is that they might both be subject to the same economic influences.

Note that the multiplication rule applies to events involving independent random vari- ables. For example, if X and Y are independent, then P(X = 10 and Y = 15) equals the prod- uct P(X = 10)P(Y = 15).

It is probably fair to say that most events in the real world are not truly indepen- dent. However, because of the simplified multiplication rule for independent events, many mathematical models assume that events are independent; the math is much easier with this assumption. The question then is whether the results from such a model are believ- able. This depends on how unrealistic the independence assumption really is.

5-2e Equally Likely Events Much of what you know about probability is probably based on situations where outcomes are equally likely. These include flipping coins, throwing dice, drawing balls from urns, and other random mechanisms that are often discussed in introductory probability books. For example, suppose an urn contains 20 red marbles and 10 blue marbles. You plan to randomly select five marbles from the urn, and you are interested, say, in the probability of selecting at least three red marbles. To find this probability, you argue that every possible set of five marbles is equally likely to be chosen. Then you count the number of sets of five marbles that contain at least three red marbles, you count the total number of sets of five marbles that could be selected, and you set the desired probability equal to the ratio of these two counts.

Multiplication Rule for Independent Events

P1A and B2 5 P1A2P1B2 (5.5)

When two events are probabilistically independent, the multiplication rule simpli- fies to Equation (5.5). This follows by substituting P(A) for P1AuB2 in the multiplication rule, which is allowed because of independence. In words, the probability that both events occur is the product of their individual probabilities.

09953_ch05_ptg01_183-241.indd 191 04/03/19 12:25 PM

1 9 2 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

It is true that many probabilities, particularly in games of chance, can be calculated by using an equally likely argument. It is also true that probabilities calculated in this way satisfy all of the rules of probability. However, many probabilities, especially those in business situations, cannot be calculated by equally likely arguments for the simple reason that the possible outcomes are not equally likely. For example, just because you are able to identify five possible scenarios for a company’s future, there is probably no reason what- soever to conclude that each scenario has probability 1/5.

The bottom line is that there is no need in this book to discuss complex counting rules for equally likely outcomes because most outcomes in the business world are not equally likely.

5-2f Subjective Versus Objective Probabilities We now ask a very basic question: Where do the probabilities in a probability distribution come from? A complete answer to this question could lead to a chapter by itself, so we only briefly discuss the issues involved. There are essentially two distinct ways to assess probabilities, objectively and subjectively. Objective probabilities are those that can be estimated from long-run proportions, whereas subjective probabilities cannot be esti- mated from long-run proportions. Some examples will clarify this distinction.

Consider throwing two dice and observing the sum of the two sides that face up. What is the probability that the sum of these two sides is 7? You might argue as follows. Because there are 6 3 6 5 36 ways the two dice can fall, and because exactly 6 of these result in a sum of 7, the probability of a 7 is 6/36 = 1/6. This is the equally likely argument and it reduces probability to counting.

What if the dice are weighted in some way? Then the equally likely argument is no longer valid. You can, however, toss the dice many times and record the proportion of tosses that result in a sum of 7. This proportion is called a relative frequency.

The relative frequency of an event is the proportion of times the event occurs out of the number of times the random experiment is run.

A famous result called the law of large numbers states that this relative frequency, in the long run, will get closer and closer to the “true” probability of a 7. This is exactly what we mean by an objective probability. It is a probability that can be estimated as the long- run proportion of times an event occurs in a sequence of many identical experiments.

If you are flipping coins, throwing dice, or spinning roulette wheels, objective prob- abilities are certainly relevant. You don’t need a person’s opinion of the probability that a roulette wheel, say, will end up pointing to a red number; you can simply spin it many times and keep track of the proportion of times it points to a red number. However, there are many situations, particularly in business, that cannot be repeated many times—or even more than once—under identical conditions. In these situations objective proba- bilities make no sense (and equally likely arguments usually make no sense either), so you must use subjective probabilities. A subjective probability is one person’s assessment of the likelihood that a certain event will occur. We assume that the person making the assessment uses all of the information available to make the most rational assessment possible.

This definition of subjective probability implies that one person’s assessment of a probability can differ from another person’s assessment of the same probability. For exam- ple, consider the probability that the New England Patriots will win the next Super Bowl. If you ask a casual football observer to assess this probability, you will get one answer, but if you ask a person with a lot of inside information about injuries, team cohesiveness, and so on, you might get a very different answer. Because these probabilities are subjective, people with different information typically assess probabilities in different ways.

09953_ch05_ptg01_183-241.indd 192 04/03/19 12:25 PM

5-2 Probability Essentials 1 9 3

Subjective probabilities are usually relevant for unique, one-time situations. However, most situations are not completely unique; you often have some history to guide you. That is, historical relative frequencies can be factored into subjective probabilities. For exam- ple, suppose a company is about to market a new product. This product might be different in some ways from any products the company has marketed before, but it might also share some features with the company’s previous products. If the company wants to assess the probability that the new product will be a success, it will certainly analyze the unique fea- tures of this product and the current state of the market to obtain a subjective assessment. However, the company will also look at its past successes and failures with reasonably similar products. If the proportion of successes with past products was 20%, say, then this value might be a starting point in the assessment of this product’s probability of success.

All of the “given” probabilities in this chapter and later chapters can be placed some- where on the objective-to-subjective continuum, usually closer to the subjective end. An important implication of this is that these probabilities are not cast in stone; they are usu- ally only educated guesses. Therefore, it is always a good idea to run a sensitivity analysis (especially in Excel, where this is easy to do) to see how “bottom-line” results depend on the given probabilities. Sensitivity analysis is especially important in Chapter 6, when we study decision making under uncertainty.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. In a particular suburb, 30% of the households have

installed electronic security systems. a. If a household is chosen at random from this suburb,

what is the probability that this household has not installed an electronic security system?

b. If two households are chosen at random from this sub- urb, what is the probability that neither has installed an electronic security system?

2. Several major automobile producers are competing to have the largest market share for sport utility vehicles (SUVs) in the coming quarter. A professional automo- bile market analyst assesses that the odds of General Motors not being the market leader are 6 to 1. The odds against Toyota and Ford having the largest market share in the coming quarter are similarly assessed to be 12 to 5 and 8 to 3, respectively. a. Find the probability that General Motors will have the

largest market share for SUVs in the coming quarter. b. Find the probability that Toyota will have the largest

market share for SUVs in the coming quarter. c. Find the probability that Ford will have the largest

market share for SUVs in the coming quarter. d. Find the probability that some other automobile man-

ufacturer will have the largest market share for SUVs in the coming quarter.

3. The publisher of a popular financial periodical has decided to undertake a campaign in an effort to attract new subscribers. Market research analysts in this

company believe that there is a 1 in 4 chance that the increase in the number of new subscriptions resulting from this campaign will be less than 3000, and there is a 1 in 3 chance that the increase in the number of new subscriptions resulting from this campaign will be between 3000 and 5000. What is the probability that the increase in the number of new subscriptions result- ing from this campaign will be less than 3000 or more than 5000?

4. Suppose that you draw a single card from a standard deck of 52 playing cards. a. What is the probability that this card is a diamond or

club? b. What is the probability that this card is not a 4? c. Given that this card is a black card, what is the proba-

bility that it is a spade? d. Let E1 be the event that this card is a black card. Let

E2 be the event that this card is a spade. Are E1 and E2 independent events? Why or why not?

e. Let E3 be the event that this card is a heart. Let E4 be the event that this card is a 3. Are E3 and E4 indepen- dent events? Why or why not?

Level B 5. In a large accounting firm, the proportion of accountants

with MBA degrees and at least five years of professional experience is 75% as large as the proportion of accoun- tants with no MBA degree and less than five years of pro- fessional experience. Furthermore, 35% of the accountants in this firm have MBA degrees, and 45% have fewer than five years of professional experience. If one of the firm’s accountants is selected at random, what is the probability that this accountant has an MBA degree or at least five years of professional experience, but not both?

09953_ch05_ptg01_183-241.indd 193 04/03/19 12:25 PM

1 9 4 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

6. A local beer producer sells two types of beer, a regular brand and a light brand with 30% fewer calories. The company’s marketing department wants to verify that its traditional approach of appealing to local white-collar workers with light beer commercials and appealing to local blue-collar workers with regular beer commercials is indeed a good strategy. A randomly selected group of 400 local workers are questioned about their beer- drinking preferences, and the data in the file P05_06. xlsx are obtained. a. If a blue-collar worker is chosen at random from this

group, what is the probability that he/she prefers light beer (to regular beer or no beer at all)?

b. If a white-collar worker is chosen at random from this group, what is the probability that he/she prefers light beer (to regular beer or no beer at all)?

c. If you restrict your attention to workers who like to drink beer, what is the probability that a randomly selected blue-collar worker prefers to drink light beer?

d. If you restrict your attention to workers who like to drink beer, what is the probability that a randomly selected white-collar worker prefers to drink light beer?

e. Does the company’s marketing strategy appear to be appropriate? Explain why or why not.

7. Suppose that two dice are tossed. For each die, it is equally likely that 1, 2, 3, 4, 5, or 6 dots will turn up. Let S be the sum of the two dice. a. What is the probability that S will be 5 or 7? b. What is the probability that S will be some number

other than 4 or 8? c. Let E1 be the event that the first die shows a 3. Let E2 be

the event that S is 6. Are E1 and E2 independent events? d. Again, let E1 be the event that the first die shows a 3.

Let E3 be the event that S is 7. Are E1 and E3 indepen- dent events?

e. Given that S is 7, what is the probability that the first die showed 4 dots?

f. Given that the first die shows a 3, what is the probabil- ity that S is an even number?

5-3 Probability Distribution of a Random Variable We now discuss one of the most important concepts in this book, the probability distribu- tion of a random variable.

There are really two types of random variables: discrete and continuous. A discrete random variable has only a finite number of possible values, whereas a continuous ran- dom variable has a continuum of possible values.1 Usually a discrete distribution results from a count, whereas a continuous distribution results from a measurement. For example, the number of children in a family is clearly discrete, whereas the amount of rain this year in San Francisco is clearly continuous.

Concept of Probability Distribution

A probability distribution describes the uncertainty of a numerical outcome. It is not based, at least not directly, on a data set of the type discussed in the previ- ous chapters. Instead, it is a list of all possible outcomes and their corresponding probabilities.

Fundamental Insight

Mathematically, there is an important difference between discrete and continuous probability distributions. Specifically, a proper treatment of continuous distributions, anal- ogous to the treatment we provide in this section, requires calculus—which we do not pre- sume for this book. Therefore, we discuss only discrete distributions in this section. Later in the book, we often use continuous distributions, particularly the bell-shaped normal distribution, but we simply state their properties without deriving them mathematically.

1 Actually, a more rigorous discussion allows a discrete random variable to have an infinite number of possible values, such as all positive integers.

09953_ch05_ptg01_183-241.indd 194 04/03/19 12:25 PM

5-3 Probability Distribution of a Random Variable 1 9 5

The essential properties of a discrete random variable and its associated probability dis- tribution are quite simple. We discuss them in general and then analyze a numerical example.

Let X be a random variable. To specify the probability distribution of X, we need to specify its possible values and their probabilities. We assume that there are k possible values, denoted x1, x2, …, xk. The probability of a typical value xi is denoted in one of two ways, either P(X = xi) or p(xi). The first is a reminder that this probability involves the ran- dom variable X, whereas the second is a shorthand notation. Probability distributions must satisfy two criteria: (1) the probabilities must be nonnegative, and (2) they must sum to 1. In symbols, we must have

ak i 5 1

p1xi2 5 1, p1xi2 $ 0

This is basically all there is to it: a list of possible values and a list of associated prob- abilities that sum to 1. It is also sometimes useful to calculate cumulative probabilities. A cumulative probability is the probability that the random variable is less than or equal to some particular value. For example, assume that 10, 20, 30, and 40 are the possible values of a random variable X, with corresponding probabilities 0.15, 0.25, 0.35, and 0.25. Then a typical cumulative probability is P1X # 302. From the addition rule it can be calculated as

P1X # 302 5 P1X 5 102 1 P1X 5 202 1 P1X 5 302 5 0.75 The point is that the cumulative probabilities are determined by the individual

probabilities.

5-3a Summary Measures of a Probability Distribution It is often convenient to summarize a probability distribution, discrete or continuous, with two or three well-chosen numbers. The first of these is the mean, often denoted m. It is also called the expected value of X and denoted E(X) (for expected X). The mean of a dis- crete distribution is a weighted sum of the possible values, weighted by their probabilities, as shown in Equation (5.6). In much the same way that an average of a set of numbers indicates “central location,” the mean indicates the “center” of the probability distribution.

Usually, capital letters toward the end of the alphabet, such as X, Y, and Z, are used to denote random variables.

A discrete probability distribution is a set of possible values and a corresponding set of probabilities that sum to 1.

Mean of a Probability Distribution, m

m 5 E1X2 5 ak i 5 1

xi p1xi2 (5.6)

To measure the variability in a distribution, we calculate its variance or standard devi- ation. The variance of a discrete distribution, denoted by s2 or Var1X2, is a weighted sum of the squared deviations of the possible values from the mean, where the weights are again the probabilities. This is shown in Equation (5.7). As in Chapter 2, the variance is expressed in the square of the units of X, such as dollars squared. Therefore, a more nat- ural measure of variability is the standard deviation, denoted by s or Stdev1X2. It is the square root of the variance, as indicated by Equation (5.8).

Variance of a Probability Distribution, s2

s2 5 Var1X2 5 ak i 3xi 2 E1X242 p1xi2 (5.7)

09953_ch05_ptg01_183-241.indd 195 04/03/19 12:25 PM

1 9 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

Equation (5.7) is useful for understanding variance as a weighted average of squared deviations from the mean. However, the following is an equivalent formula for variance and is somewhat easier to implement in Excel. (It can be derived with straightforward algebra.) In words, you find the weighted average of the squared values, weighted by their probabilities, and then subtract the square of the mean.

Standard Deviation of a Probability Distribution, s

s 5 Stdev1X2 5 !Var1X2 (5.8)

Variance (computing formula)

s2 5 ak i 5 1

xi 2 p1xi2 2 m2 (5.9)

We now consider a typical example.

EXAMPLE

5.2 MARKET RETURN SCENARIOS FOR THE NATIONAL ECONOMY An investor is concerned with the market return for the coming year, where the market return is defined as the percent- age gain (or loss, if negative) over the year. The investor believes there are five possible scenarios for the national econ- omy in the coming year: rapid expansion, moderate expansion, no growth, moderate contraction, and serious contraction. Furthermore, she has used all of the information avail- able to her to estimate that the market returns for these scenarios are, respectively, 23%, 18%, 15%, 9%, and 3%. That is, the possible returns vary from a high of 23% to a low of 3%. Also, she has assessed that the probabilities of these outcomes are 0.12, 0.40, 0.25, 0.15, and 0.08. Use this information to describe the probability distribution of the market return.

Objective To compute the mean, variance, and standard deviation of the probability distribution of the market return for the coming year.

Solution To make the connection between the general notation and this particular example, let X denote the market return for the coming year. Then each possible economic scenario leads to a possible value of X. For example, the first possible value is x1 5 23%, and its probability is p1x12 5 0.12. These values and probabilities appear in columns B and C of Figure 5.3. (See the file Market Return Finished.xlsx.) Note that the five probabilities sum to 1, as they should. This probability distribution implies, for example, that the probability of a market return at least as large as 18% is 0.12 1 0.40 5 0.52 because it could occur as a result of rapid or moderate expansion of the economy. Similarly, the probability that the market return is 9% or less is 0.15 1 0.08 5 0.23 because this could occur as a result of moderate or serious contraction of the economy.

The summary measures of this probability distribution appear in the range B11:B13. They can be calculated with the fol- lowing steps. (Note that the formulas make use of the range names listed in the figure.)

In reality, there is a continuum of possible returns. The assumption of only five possible returns is clearly an approximation to reality, but such an approxi- mation is often useful.

09953_ch05_ptg01_183-241.indd 196 04/03/19 12:25 PM

Procedure for Calculating Summary Measures 1. Mean return. Calculate the mean return in cell B11 with the formula

5SUMPRODUCT(Market_return,Probability)

Figure 5.3 Probability Distribution of Market Returns

1 2 3 4 5 6 7 8 9

10 11 12 13

A B C D E F G Mean, variance, and standard deviation of the market return �an�e names �sed

Market_return =$C$4:$C$8 Economic outcome Probability Market return Sq dev from mean Mean =$B$11 Rapid Expansion 0.005929 Probability =$B$4:$B$8 Moderate Expansion 0.000729

0.000009 Sq_dev_from_mean =$D$4:$D$8

No Growth Stdev Moderate Contraction 0.003969 Variance =$B$12

=$B$13

Serious Contraction

Summary measures of return Mean Variance 0.002811

15.3%

5.3% 0.002811 Quick alternative formula

5.3%Stdev

0.015129

0.12 0.23 0.40 0.18

0.150.25 0.15 0.09

0.030.08

Excel’s SUMPRODUCT Function

Excel’s SUMPRODUCT function is a gem, and you should use it whenever possible. It takes (at least) two argu- ments, which must be ranges of exactly the same size and shape. It sums the products of the values in these rang- es. For example, = SUMPRODUCT 1A1:A3,B1:B32 is equivalent to the formula = A1*B1 1 A2*B2 1 A3*B3. If the ranges contain only a few cells, there isn’t much advantage to using SUMPRODUCT, but when the ranges are large, such as A1:A100 and B1:B100, SUMPRODUCT is the only viable option.

Excel Tip

This formula illustrates the general rule in Equation (5.6): The mean is the sum of products of possible values and probabilities.

2. Squared deviations. To get ready to compute the variance from Equation (5.7), calculate the squared deviations from the mean by entering the formula

5(C4-Mean)^2

in cell D4 and copying it down through cell D8. 3. Variance. Calculate the variance of the market return in cell B12 with the formula

5SUMPRODUCT(Sq_dev_from_mean,Probability)

This illustrates the general formula for variance in Equation (5.7). The variance is always a sum of products of squared deviations from the mean and probabilities. Alternatively, you can skip the calculation of the squared deviations from the mean and use Equation (5.9) directly. This is done in cell C12 with the formula

5SUMPRODUCT(Market_return,Market_return,Probability)-Mean^2

By entering the Market_return range twice in this SUMPRODUCT formula, you get the squares. From now on, we will use this simplified formula for variance and dispense with squared deviations from the mean. But regardless of how it is calculated, you should remember the essence of variance: It is a weighted average of squared deviations from the mean.

4. Standard deviation. Calculate the standard deviation of the market return in cell B13 with the formula

5SQRT(Variance)

As always, range names are not required, but they make the Excel formulas easier to read. You can use them or omit them, as you wish.

5-3 Probability Distribution of a Random Variable 1 9 7

09953_ch05_ptg01_183-241.indd 197 04/03/19 12:25 PM

You can see that the mean return is 15.3% and the standard deviation is 5.3%. What do these measures really mean? First, the mean, or expected, return does not imply that the most likely return is 15.3%, nor is this the value that the investor “expects” to occur. In fact, the value 15.3% is not even a possible market return, at least not according to the model. You can understand these measures better in terms of long-run averages. Specifically, if you could imagine the coming year being repeated many times, each time using the probability distribution in columns B and C to generate a market return, then the average of these market returns would be close to 15.3%, and their standard deviation—calculated as in Chapter 2—would be close to 5.3%.

Before leaving this section, we emphasize a key point, a point that is easy to forget with all the details. The whole point of discussing probability and probability distributions, especially in the context of business problems, is that uncertainty is often a key factor, and you cannot simply ignore it. The mean return in this example is 15.3%. However, it would be far from realistic to treat the actual return as a sure 15.3%, with no uncertainty. If you did this, you would be ignoring the uncertainty completely, and it is often the uncertainty that makes business problems interesting—and adds risk. Therefore, to model such problems in a realistic way, you must deal with probability and probability distributions.

2 This section is somewhat advanced and can be omitted. It won’t be used in the rest of the book.

Conditional Mean Formula

E1X2 5 ak i 5 1

Ei1X2pi (5.10)

Conditional Variance Formula

Var 1X2 5 ak i 5 1

5Vari1X2 1 3Ei1X2426pi 2 3E 1X242 (5.11)

1 9 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5-3b Conditional Mean and Variance2

There are many situations where the mean and variance of a random variable depend on some external event. In this case, you can condition on the outcome of the external event to find the overall mean and variance (or standard deviation) of the random variable.

It is best to motivate this with an example. Consider the random variable X, repre- senting the percentage change in the price of stock A from now to a year from now. This change is driven partly by circumstances specific to company A, but it is also driven partly by the economy as a whole. In this case, the outcome of the economy is the external event. Let’s assume that the economy in the coming year will be awful, stable, or great with probabilities 0.20, 0.50, and 0.30, respectively. (Of course, these are subjective probabili- ties.) In addition, we make the following assumptions.

• Given that the economy is awful, the mean and standard deviation of X are 220% and 30%.

• Given that the economy is stable, the mean and standard deviation of X are 5% and 20%.

• Given that the economy is great, the mean and standard deviation of X are 25% and 15%.

Each of these statements is a statement about X conditional upon the economy. What can we say about the unconditional mean and standard deviation of X? That is, what are the mean and standard deviation of X before we learn the state of the economy? The answers come from Equations (5.10) and (5.11). In the context of the example, pi is the probability of economy state i, and Ei1X2 and Vari1X2 are the mean and variance of X, given that econ- omy state i occurs.

09953_ch05_ptg01_183-241.indd 198 04/03/19 12:25 PM

5-3 Probability Distribution of a Random Variable 1 9 9

In the example, the mean percentage change in the price of stock A, from Equation (5.10), is

E1X2 5 0.21220%2 1 0.515%2 1 0.3125%2 5 6% To calculate the standard deviation of X, first use Equation (5.11) to calculate the variance, and then take its square root. The variance is

Var1X2 5 50.23130%22 1 1220%224 1 0.53120%22 1 15%224

1 0.33115%22 1 125%2246 2 16%22 5 0.06915 Taking the square root gives

Stdev1X2 5 !0.06915 5 26.30% Of course, these calculations can be done easily in Excel. See the file Stock Price and Economy Finished.xlsx for details.

The point of this example is that it is often easier to assess the uncertainty of some random variable X by conditioning on every possible outcome of some external event like the economy. However, before that outcome is known, the relevant mean and standard deviation of X are those calculated from Equations (5.10) and (5.11). In this particular example, before you know the state of the economy, the relevant mean and standard devia- tion of the change in the price of stock A are 6% and 26.3%, respectively.

the random variable X, the number of months from now it will take to complete this project: 2, 2.5, 3, and 3.5. The manager currently thinks that the probabilities of these four possibilities are in the ratio 1 to 2 to 4 to 2. That is, X 5 2.5 is twice as likely as X 5 2, X 5 3 is twice as likely as X 5 2.5, and X 5 3.5 is half as likely as X 5 3. a. Find the probability distribution of X. b. What is the probability that this project will be com-

pleted in less than three months from now? c. What is the probability that this project will not be

completed on time? d. What is the expected completion time (in months) of

this project from now? e. How much variability (in months) exists around the

expected value you found in part d?

Level B 11. The National Football League playoffs are just about to

begin. Because of their great record in the regular sea- son, the Steelers get a bye in the first week of the play- offs. In the second week, they will play the winner of the game between the Ravens and the Patriots. A football expert estimates that the Ravens will beat the Patriots with probability 0.45. This same expert estimates that if the Steelers play the Ravens, the mean and standard deviation of the point spread (Steelers points minus Ravens points) will be 6.5 and 10.5, whereas if the Steelers play the Patriots, the mean and standard devi- ation of the point spread (Steelers points minus Patriots points) will be 3.5 and 12.5. Find the mean and standard deviation of the point spread (Steelers points minus their opponent’s points) in the Steelers game.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 8. Consider a random variable with the following proba-

bility distribution: P1X 5 02 5 0.1, P1X 5 12 5 0.2, P1X 5 22 5 0.3, P1X 5 32 5 0.3, and P1X 5 42 5 0.1. a. Find P1X # 22. b. Find P11 6 X # 32. c. Find P1X 7 02. d. Find P1X 7 3|X 7 22. e. Find the expected value of X. f. Find the standard deviation of X.

9. A study has shown that the probability distribu- tion of X, the number of customers in line (including the one being served, if any) at a checkout counter in a department store, is given by P1X 5 02 5 0.25, P1X 5 12 5 0.25, P1X 5 22 5 0.20, P1X 5 32 5 0.20, and P1 $ 42 5 0.10. Consider a newly arriving cus- tomer to the checkout line. a. What is the probability that this customer will not

have to wait behind anyone? b. What is the probability that this customer will have to

wait behind at least one customer? c. On average, the newly arriving customer will have to

wait behind how many other customers? 10. A construction company has to complete a project no

later than three months from now or there will be sig- nificant cost overruns. The manager of the construction company believes that there are four possible values for

09953_ch05_ptg01_183-241.indd 199 04/03/19 12:25 PM

2 0 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5-4 The Normal Distribution In the rest of this chapter, we discuss specific probability distributions, starting with the single most important distribution in statistics, the normal distribution. It is a continuous distribution and is the basis of the familiar symmetric bell-shaped curve. Any particular normal distribution is specified by its mean and standard deviation. By changing the mean, the normal curve shifts to the right or left. By changing the standard deviation, the curve becomes more or less spread out. Therefore, there are really many normal distributions, not just a single one. We say that the normal distribution is a two-parameter family, where the two parameters are the mean and the standard deviation.

5-4a Continuous Distributions and Density Functions We first take a moment to discuss continuous probability distributions in general. In the previous section we discussed discrete distributions, characterized by a list of possible values and their probabilities. The same basic idea holds for continuous distributions such as the normal distribution, but the mathematics is more complex. Now instead of a list of possible values, there is a continuum of possible values, such as all values between 0 and 100 or all values greater than 0. Instead of assigning probabilities to each individual value in the continuum, the total probability of 1 is spread over this continuum. The key to this spreading is called a density function, which acts like a histogram. The higher the value of the density function, the more likely this region of the continuum is.

12. The “house edge” in any game of chance is defined as

E1player’s loss on a bet2 Size of player’s loss on a bet

For example, if a player wins $10 with probability 0.48 and loses $10 with probability 0.52 on any bet, the house edge is

2 31010.482 2 1010.5224 10

5 0.04

Give an interpretation to the house edge that relates to how much money the house is likely to win on average. Which do you think has a larger house edge: roulette or sports gambling? Why?

A density function, usually denoted by f1x2, specifies the probability distribu- tion of a continuous random variable X. The higher f1x2 is, the more likely x is. Also, the total area between the graph of f1x2 and the horizontal axis, which represents the total probability, is equal to 1. Finally, f1x2 is nonnegative for all possible values of X.

As an example, consider the density function shown in Figure 5.4. (This is not a nor- mal density function.) It indicates that all values in the continuum from 25 to 100 are pos- sible, but that the values near 70 are most likely. (This density function might correspond to scores on an exam.) More specifically, because the height of the density at 70 is approx- imately twice the height of the curve at 84 or 53, a value near 70 is approximately twice as likely as a value near 84 or a value near 53. In this sense, the height of the density function indicates relative likelihoods.

Probabilities are found from a density function as areas under the curve. For exam- ple, the area of the designated region in Figure 5.5 represents the probability of a score between 65 and 75. Also, the area under the entire curve is 1 because the total probability of all possible values is always 1. Fortunately, Excel functions have been developed to find these areas—without the need for bulky tables. We take advantage of these Excel func- tions in the rest of this chapter and in later chapters.

For continuous distributions, probabilities are areas under the density function. These probabilities can often be calculated with Excel functions.

09953_ch05_ptg01_183-241.indd 200 04/03/19 12:25 PM

5-4 The Normal Distribution 2 0 1

Figure 5.4 A Skewed Density Function

3.5

2.5

1.5

0.5

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99

As in the previous chapter, the mean is a measure of central tendency of the distribu- tion, and the standard deviation (or variance) measures the variability of the distribution. We will list their values for the normal distribution and any other continuous distributions where we need them. By the way, the mean for the (nonnormal) density in Figure 5.4 is slightly less than 70—it is always to the left of the peak for a left-skewed distribution and to the right of the peak for a right-skewed distribution—and the standard deviation is approximately 15.

5-4b The Normal Density Function The normal distribution is a continuous distribution with possible values ranging over the entire number line—from “minus infinity” to “plus infinity.” However, only a rela- tively small range has much chance of occurring. The normal density function is actually quite complex, in spite of its “nice” bell-shaped appearance. For the sake of completeness, we list the formula for the normal density function in Equation (5.12). Here, m and s are the mean and standard deviation of the distribution.

Figure 5.5 Probability as the Area Under the Density

3.5

2.5

1.5

0.5

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99

Area under curve is probability of being between 65 and 75

Normal Density Function

f 1x2 5 1!2pse 2 1x 2 m22/12s22 for 2 ` 6 x 6 1 ` (5.12)

The curves in Figure 5.6 illustrate several normal density functions for different values of m and s. The mean m can be any number: negative, positive, or zero. As you can see, the effect of increasing or decreasing the mean m is to shift the curve to the right or the left. On the other hand, the standard deviation s must be a positive number. It controls the

09953_ch05_ptg01_183-241.indd 201 04/03/19 12:25 PM

2 0 2 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

spread of the normal curve. When s is small, the curve is more peaked; when s is large, the curve is more spread out. For shorthand, we use the notation N1m, s2 to refer to the normal distribution with mean m and standard deviation s. For example, N122, 12 refers to the normal distribution with mean 22 and standard deviation 1.

Figure 5.6 Several Normal Density Functions

5-4c Standardizing: Z-Values There are infinitely many normal distributions, one for each pair m and s. We single out one of these for special attention, the standard normal distribution. The standard normal distribution has mean 0 and standard deviation 1, so we denote it by N10, 12. It is also commonly referred to as the Z distribution. Suppose the random variable X is normally distributed with mean m and standard deviation s. We define the random variable Z by Equation (5.13). This operation is called standardizing. That is, to standardize a variable, you subtract its mean and then divide the difference by the standard deviation. When X is normally distributed, the standardized variable is N10, 12.

Why the Normal Distribution?

The normal density in Equation (5.12) is certainly not very intuitive, so why is the normal distribution the basis for so much of statistical theory? One reason is practical. Many histograms based on real data resemble the bell-shaped nor- mal curve to a remarkable extent. Granted, not all histograms are symmetric and bell shaped, but a surprising number are. Another reason is theoretical. In spite of the complexity of Equation (5.12), the normal distribution has many appealing properties that have enabled researchers to build the rich statistical theory that finds widespread use in business, the sciences, and other fields.

Fundamental Insight

Standardizing a Normal Random Variable

Z 5 X 2 m

s (5.13)

One reason for standardizing is to measure variables with different means and/or stan- dard deviations on a single scale. For example, suppose several sections of a college course are taught by different instructors. Because of differences in teaching methods and grading

09953_ch05_ptg01_183-241.indd 202 04/03/19 12:25 PM

5-4 The Normal Distribution 2 0 3

procedures, the distributions of scores in these sections might differ. However, if each instruc- tor calculates his or her mean and standard deviation and then calculates a Z-value for each student, the distributions of the Z-values should be approximately the same in each section.

It is easy to interpret a Z-value. It is the number of standard deviations to the right or the left of the mean. If Z is positive, the original value (in this case, the original score) is to the right of the mean; if Z is negative, the original score is to the left of the mean. For example, if the Z -value for some student is 2, then this student’s score is two standard deviations above the mean. If the Z-value for another student is 20.5, then this student’s score is half a standard deviation below the mean. We illustrate Z-values in Example 5.1.

EXAMPLE

5.3 STANDARDIZING RETURNS FROM MUTUAL FUNDS The annual returns for 30 mutual funds appear in Figure 5.7. (See the file Standardizing Finished.xlsx.) Find and interpret the Z-values of these returns.

Objective To use Excel to standardize annual returns of various mutual funds.

A B C D E F

Standardizing mutual fund returns

Summary statistics from values below

Mean

Stdev

Fund Annual return Z value

0.091

0.047

0.000

1.000

0.007

0.080

0.082

0.123

0.022

0.054

0.094

–1.8047

–0.2363

Mean

Stdev

=$B$4

=$B$5

–0.1934

0.6875

–1.4824

–0.7949

0.0645

Range names used

0.078 –0.279336 29

0.066 –0.537137 30

Figure 5.7 Mutual Fund Returns and Z-Values

Solution The 30 annual returns appear in column B of Figure 5.7. Their mean and standard deviation are calculated in cells B4 and B5 with the AVERAGE and STDEV.S functions. The corresponding Z-values are calculated in column C by entering the formula

5(B8-Mean)/Stdev

in cell C8 and copying it down column C. (Note that Mean and Stdev are range names for cells B4 and B5.) The Z-values in Figure 5.7 range from a low of 21.80 to a high of 2.19. Specifically, the return for stock 1 is about 1.80

standard deviations below the mean, whereas the return for fund 17 is about 2.19 standard deviations above the mean. As you will see shortly, these values are typical: Z-values are usually in the range from 22 to 12 and values beyond −3 or 13 are very uncommon. (Recall the empirical rules for interpreting standard deviation discussed in Chapter 2.) Also, the Z-values automatically have mean 0 and standard deviation 1, as you can see in cells C4 and C5 by using the AVERAGE and STDEV.S functions on the Z-values in column C.

09953_ch05_ptg01_183-241.indd 203 04/03/19 12:25 PM

2 0 4 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5-4d Normal Tables and Z-Values3 A common use for Z-values and the standard normal distribution is in calculating proba- bilities and percentiles by the traditional method. This method is based on a table of the standard normal distribution found in many statistics textbooks. Such a table is given in Figure 5.8. The body of the table contains probabilities. (The entire body of this table was generated by the single copyable formula shown.) The left and top margins contain possible values. Specifically, suppose you want to find the probability that a standard normal random variable is less than 1.35. You locate 1.3 along the left and 0.05—the second decimal in 1.35—along the top, and then read into the table to find the probability 0.9115. In words, the probability is about 0.91 that a standard normal random variable is less than 1.35.

Figure 5.8 Table of Normal Probabilities

z 0.00 0.01 0.02 0.03 0.05 0.06 0.07 0.08 0.09 0.0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359 0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753 0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141 0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517 0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879 0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224 0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549 0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852 0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133

0.8159 0.8186 0.8212 0.8238 0.8264 0.8365 0.8389 1.0 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621 1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830 1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015 1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177 1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319 1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441 1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545 1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633 1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706 1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767 2.0 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817 2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857 2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890 2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916 2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936 2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952 2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964 2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974 2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981 2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986 3.0 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990 3.1 0.9990 0.9991 0.9991 0.9991 0.9992 0.9992 0.9992 0.9992 0.9993 0.9993 3.2 0.9993 0.9993 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995 3.3 0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9996 0.9996 0.9996 0.9997 3.4 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998

A B C D E G H I J K

0.9

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

=NORM.S.DIST($A11+F$1, TRUE) NORM.S.DIST(z, cumulative)

0.04 F

3 If you intend to rely on Excel functions for normal calculations (and we hope you do!), you can skip this subsection.

09953_ch05_ptg01_183-241.indd 204 04/03/19 12:25 PM

5-4 The Normal Distribution 2 0 5

Alternatively, if you are given a probability, you can use the table to find the value with this much probability to the left of it under the standard normal curve. This is called a percentile calculation. For example, if the probability is 0.75, you can find the 75th percentile by locating the probability in the table closest to 0.75 and then reading to the left and up. With interpolation, the required value is approximately 0.675. In words, the probability of being to the left of 0.675 under the standard normal curve is approximately 0.75.

There are some obvious drawbacks to using the standard normal table for probability calculations. The first is that there are holes in the table—interpolation is often necessary. A second drawback is that the standard normal table takes different forms in different textbooks. These differences are rather minor, but they can easily cause confusion. Finally, the table requires you to perform calculations, where errors are easy to make.

The Excel functions discussed next make the whole procedure much easier and less error-prone.

5-4e Normal Calculations in Excel Two types of calculations are typically made with normal distributions: finding probabil- ities and finding percentiles. Excel has functions for both of these. The functions used for normal probability calculations are NORM.DIST and NORM.S.DIST. The main dif- ference between these is that the one with the “S” (for standard) applies only to N10, 12 calculations, whereas NORM.DIST applies to any normal distribution. On the other hand, percentile calculations that take a probability and return a value are often called inverse calculations. Therefore, the Excel functions for these are named NORM.INV and NORM.S.INV. Again, the “S” in the second of these indicates that it applies to the standard normal distribution.

The NORM.DIST and NORM.S.DIST functions return left-tail probabilities, such as the probability that a normally distributed variable is less than 35. The syntax for these functions is

5NORM.DIST(x,m,s,TRUE)

and

5NORM.S.DIST (x,TRUE)

Here, x is a number you supply, and m and s are the mean and standard deviation of the normal distribution. The last argument in the NORM.DIST function, TRUE (or 1), is used to obtain the cumulative normal probability, the type usually required. (This TRUE/FALSE argument is a nuisance to remember, but it is necessary. If the last argument is FALSE, or 0, the function returns the height of the density at x, which is not usually what you want.) The NORM.S.DIST function takes only two arguments (because m and s are known to be 0 and 1), so it is easier to use—when it applies. (Note that the “old” NORMSDIST function doesn’t require the last TRUE/FALSE argument, but the newer NORM.S.DIST function, introduced in Excel 2010, does.)

The NORM.INV and NORM.S.INV functions return values for user-supplied probabilities. For example, if you supply the probability 0.95, these functions return the 95 th percentile. Their syntax is

=NORM.INV(p,m,s)

and

=NORM.S.INV(p)

where p is a probability you supply. These are analogous to the NORM.DIST and NORM.S.DIST functions except there is no final TRUE/FALSE argument in the “INV” functions.

09953_ch05_ptg01_183-241.indd 205 04/03/19 12:25 PM

2 0 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

We illustrate these Excel functions in the following example.4

Old Normal Functions

Prior to Excel 2010, the normal functions were NORMDIST, NORMSDIST, NORMINV, and NORMSINV. They are almost the same as the new functions, the ones with periods in their names, except that NORMSDIST does not take a last TRUE/FALSE argument. Microsoft kept these old functions for backward compatibility, so you can continue to use them if you prefer.

Excel Tip

Probability and Percentile Calculations

There are two basic types of calculations involving probability distributions, nor- mal or otherwise. In a probability calculation, you provide a possible value, and you ask for the probability of being less than or equal to this value. In a percentile calculation, you provide a probability, and you ask for the value that has this probability to the left of it. Excel’s statistical functions use DIST functions to perform probability calculations and INV (for inverse) functions to perform per- centile calculations.

Fundamental Insight

EXAMPLE

5.4 NORMAL CALCULATIONS IN EXCEL Use Excel to calculate the following probabilities and percentiles for the standard normal distribution: (a) P1Z 6 2 22, (b) P1Z 7 12, (c) P120.4 6 Z 6 1.62 , (d) the 5th percentile, (e) the 75th percentile, and (f) the 99th percentile. Then for the N175, 82 distribution, find the following probabilities and percentiles: (a) P1X 6 702, (b) P1X 7 732, (c) P175 6 X 6 852, (d) the 5th percentile, (e) the 60th percentile, and (f) the 97th percentile.

Objective To calculate probabilities and percentiles for standard normal and general normal distributions in Excel.

Solution The solution appears in Figure 5.9. (See the file Normal Calculations Finished.xlsx.) The N10, 12 calculations are in rows 7 through 14; the N175, 82 calculations are in rows 23 through 30. For your convenience, the formulas used in column B are spelled out in column D (as labels). Note that the standard normal calculations use the normal functions with the “S” in the middle; the rest use the normal functions without the “S”—and require more arguments.

Note the following for normal probability calculations:

• For “less than” probabilities, use NORM.DIST or NORM.S.DIST directly. (See rows 7 and 23.)

• For “greater than” probabilities, subtract the NORM.DIST or NORM.S.DIST function from 1. (See rows 8 and 24.) • For “between” probabilities, subtract the two NORM.DIST or NORM.S.DIST functions. For example, in row 9 the proba-

bility of being between 20.4 and 1.6 is the probability of being less than 1.6 minus the probability of being less than 20.4.

4 Actually, we already illustrated the NORM.S.DIST function; it was used to create the body of Figure 5.8. In other words, you can use it to build your own normal probability table.

09953_ch05_ptg01_183-241.indd 206 04/03/19 12:25 PM

The percentile calculations are even more straightforward. In most percentile problems you want to find the value with a cer- tain probability to the left of it. In this case you use the NORM.INV or NORM.S.INV function with the specified probability as the first argument. See rows 12 through 14 and 28 through 30.

Note that when you are calculating probabilities for any continuous distribution, including the normal distribution, there is no need to distinguish between “less than” and “less than or equal to” events, or between “greater than” and “greater than or equal to” events. The reason is that there is no positive probability of being equal to any particular value. However, as you will see when we discuss the binomial distribution, this is not true of discrete distributions.

There are a couple of variations of percentile calculations. First, suppose you want the value with probability 0.05 to the right of it. This is the same as the value with probability 0.95 to the left of it, so you use NORM.INV or NORM.S.INV with probability argument 0.95. For example, the value with probability 0.4 to the right of it in the N175, 82 distribution is the 60th percentile, 77.027. (See cell B29 in Figure 5.6.)

As a second variation, suppose you want to find an interval of the form 2x to x, for some positive number x, with (1) probability 0.025 to the left of 2x, (2) probability 0.025 to the right of x, and (3) probability 0.95 between 2x and x. This is a very common problem in statistical inference. In general, you want a probability (such as 0.95) to be in the middle of the interval so that half of the remaining probability (0.025) is in each of the tails. (See Figure 5.10.) Then the required x

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

A B C D E F G H I

Range Probability Less than –2 0.0228 =NORM.S.DIST(–2,TRUE) Greater than 1 0.1587 =1-NORM.S.DIST(1,TRUE) Between –0.4 and 1.6 0.6006 =NORM.S.DIST(1.6,TRUE)-NORM.S.DIST(–0.4,TRUE)

5th –1.645 =NORM.S.INV(0.05)

=NORM.S.INV(0.99) =NORM.S.INV(0.75)0.67475th

2.32699th

Range names used: MeanMean StdevStdev

Range Probability Less than 70 0.2660 =NORM.DIST(70,Mean,Stdev,TRUE) Greater than 73 0.5987 =1-NORM.DIST(73,Mean,Stdev,TRUE) Between 75 and 85 0.3944 =NORM.DIST(85,Mean,Stdev,TRUE)-NORM.DIST(75,Mean,Stdev,TRUE)

Normal probability calculations

Examples with standard normal

Probability calculations

Percentiles

Examples with nonstandard normal

Probability calculations

Percentiles

61.841 77.027 90.046

5th 60th

=NORM.INV(0.97,Mean,Stdev) =NORM.INV(0.6,Mean,Stdev) =NORM.INV(0.05,Mean,Stdev)

97th

= $B$18 = $B$19

75 8

Figure 5.9 Normal Calculations with Excel Functions

Figure 5.10 Typical Normal Probabilities

5-4 The Normal Distribution 2 0 7

09953_ch05_ptg01_183-241.indd 207 04/03/19 12:25 PM

can be found with NORM.INV or NORM.S.INV, using probability argument 0.975, because there must be a total probabil- ity of 0.975 to the left of x.

For example, if the relevant distribution is the standard normal, the required value of x is 1.96, found with the function NORM.S.INV(0.975). Similarly, if you want probability 0.90 in the middle and probability 0.05 in each tail, the required x is 1.645, found with the function NORM.S.INV(0.95). Remember these two numbers, 1.96 and 1.645. They occur frequently in statistical applications.

2 0 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5-4f Empirical Rules Revisited We introduced three empirical rules in Chapter 2 that apply to many data sets. Namely, about 68% of the data fall within one standard deviation of the mean, about 95% fall within two standard deviations of the mean, and almost all fall within three standard deviations of the mean. For these rules to hold with real data, the distribution of the data must be at least approximately symmetric and bell-shaped. Let’s look at these rules more closely.

Suppose X is normally distributed with mean m and standard deviation s. To per- form a probability calculation on X, it is useful to first standardize X and then perform the calculation on the standardized variable Z. Specifically, we will find the probability that X is within k standard deviations of its mean for k 5 1, k 5 2, and k 5 3. In general, this probability is P1m 2 ks 6 X 6 m 1 ks2. But by standardizing the values m 2 ks and m 1 ks, we obtain the equivalent probability P12k 6 Z 6 k2, where Z has a N10, 12 dis- tribution. This latter probability can be calculated in Excel with the formula

5NORM.S.DIST(k,TRUE)−NORM.S.DIST(−k,TRUE)

By substituting the values 1, 2, and 3 for k, we find the following probabilities:

P(21 < Z * 1) 5 0.6827

P(22 < Z * 2) 5 0.9545

P(23 < Z * 3) 5 0.9973

As you can see, there is virtually no chance of being beyond three standard deviations from the mean, the chances are about 19 out of 20 of being within two standard deviations of the mean, and the chances are about 2 out of 3 of being within one standard deviation of the mean. These probabilities are the basis for the empirical rules in Chapter 2. These rules more closely approximate reality as the histograms of observed data become more sym- metric and bell-shaped.

5-4g Weighted Sums of Normal Random Variables One very attractive property of the normal distribution is that if you create a weighted sum of normally distributed random variables, the weighted sum is also normally distributed. In fact, this is true even if the random variables are not independent, but we will examine only the independent case here.

Specifically, if X1 through Xn are n independent and normally distributed ran- dom variables with common mean m and common standard deviation s, then the sum X1 1 g 1 Xn is normally distributed with mean nm (sum of the means), variance ns2 (sum of the variances), and standard deviation !ns (square root of the variance). More generally, if a1 through an are any constants and each X has its own mean and standard deviation, then the weighted sum a1X1 1 g 1 an Xn is normally distributed with mean a1m1 1 g 1 an mn and variance a12s12 1 g 1 a2n s2n. You will need this fact to solve a few of the problems in this chapter. (Actually, this is the correct mean even if the X’s are not independent, but in that case, the variance of the weighted sum also includes covari- ance terms and is not shown here.)

The normal distribution is the basis for the empir- ical rules introduced in Chapter 2.

09953_ch05_ptg01_183-241.indd 208 04/03/19 12:25 PM

5-4 The Normal Distribution 2 0 9

5-4h Normal Distribution Examples In this section we apply the normal distribution to several business problems.

EXAMPLE

5.5 PERSONNEL TESTING AT ZTEL The personnel department of ZTel, a large communications company, is reconsidering its hiring policy. Each applicant for a job must take a standard exam, and the hire or no-hire decision depends at least in part on the result of the exam. The scores of all applicants have been examined closely. They are approximately normally distributed with mean 525 and standard deviation 55.

The current hiring policy occurs in two phases. The first phase separates all applicants into three categories: automatic accepts, automatic rejects, and maybes. The automatic accepts are those whose test scores are 600 or above. The automatic rejects are those whose test scores are 425 or below. All other applicants (the maybes) are passed on to a second phase where their previous job experience, special talents, and other factors are used as hiring criteria. The personnel manager at ZTel wants to calculate the percentage of applicants who are automatic accepts or rejects, given the current standards. She also wants to know how to change the standards to automatically reject 10% of all applicants and automatically accept 15% of all applicants.

Objective To determine test scores that can be used to accept or reject job applicants at ZTel.

Solution Let X be the test score of a typical applicant. Then historical data suggest that the distribution of X is N1525, 552. A probability such as P1X # 4252 can be interpreted as the probability that a typical applicant is an automatic reject, or it can be interpreted as the percentage of all applicants who are automatic rejects. Given this observation, the solution to ZTel’s problem appears in Figure 5.11. (See the file Personnel Decisions Finished.xlsx.) The probability that a typical applicant is automatically accepted is 0.0863, found in cell B10 with the formula

51–NORM.DIST(B7,Mean,Stdev,TRUE)

Similarly, the probability that a typical applicant is automatically rejected is 0.0345, found in cell B11 with the formula

Figure 5.11 Calculations for Personnel Example

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18

A B C D E F Personnel Decisions

Range names used: Mean of test scores Mean =$B$3 Stdev of test scores Stdev =$B$4

Current Policy Automatic accept point Automatic reject point

Percent accepted Percent rejected

=1-NORM.DIST(B7,Mean,Stdev,TRUE) =NORM.DIST(B8,Mean,Stdev,TRUE)

New Policy Percent accepted Percent rejected

Automatic accept point 582 =NORM.INV(1-B14,Mean,Stdev) Automatic reject point 455 =NORM.INV(B15,Mean,Stdev)

525 55

600 425

8.63% 3.45%

15% 10%

09953_ch05_ptg01_183-241.indd 209 04/03/19 12:25 PM

EXAMPLE

5.6 QUALITY CONTROL AT PAPERSTOCK PaperStock Company runs a manufacturing facility that produces a paper product. The fiber content of this prod- uct is supposed to be 20 pounds per 1000 square feet. (This is typical for the type of paper used in grocery bags, for example.) Because of random variations in the inputs to the process, however, the fiber content of a typical 1000-square-foot roll varies according to a N1m, s2 distribution. The mean fiber content 1m2 can be controlled—that is, it can be set to any desired level by adjusting an instrument on the machine. The variability in fiber content, as measured by the standard deviation s, is 0.10 pound when the process is “good,” but it sometimes increases to 0.15 pound when the machine goes “bad.” A given roll of this product must be rejected if its actual fiber content is less than 19.8 pounds or greater than 20.3 pounds. Calculate the probability that a given roll is rejected, for a setting of m 5 20, when the machine is “good” and when it is “bad.”

Objective To determine the machine settings that result in paper of acceptable quality at PaperStock Company.

Solution Let X be the fiber content of a typical roll. The distribution of X will be either N120, 0.102 or N120, 0.152, depending on the status of the machine. In either case, the probability that the roll must be rejected can be calculated as shown in Figure 5.12. (See the file Paper Machine Settings Finished.xlsx.) The formula for rejection in the “good” case appears in cell B12 and is spelled out to its right.

This is the sum of two probabilities: the probability of being to the left of the lower limit and the probability of being to the right of the upper limit. (See Figure 5.13.) A similar formula for the “bad” case appears in cell B13, using Stdev_bad in place of Stdev_good.

You can see that the probability of a rejected roll in the “good” case is 0.024; in the “bad” case it is 0.114. That is, when the standard deviation increases by 50% from 0.10 to 0.15, the percentage of rolls rejected more than quadruples, from 2.4% to 11.4%.

It is certainly possible that the true process mean and “good” standard deviation will not always be equal to the values in cells B3 and B4. Therefore, it is useful to see how sensitive the rejection probability is to these two parameters. You can do this with a two- way data table, as shown in Figure 5.12. The tabulated values show that the probability of rejection varies greatly even for small changes in the key inputs. In particular, a combi- nation of a badly centered mean and a large standard deviation can make the probability of rejection quite large.

To form this data table, enter the formula 5 B12 in cell B17, highlight the range B17:H25, and create a data table with row input cell B4 and column input cell B3.

2 1 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5NORM.DIST(B8,Mean,Stdev,TRUE)

Therefore, ZTel automatically accepts about 8.6% and rejects about 3.5% of all applicants under the current policy. To find new cutoff values that reject 10% and accept 15% of the applicants, we need the 10th and 85th percentiles of the

N1525, 552 distribution. These are 455 and 582 (rounded to the nearest integer), respectively, found in cells B17 and B18 with the formulas

5NORM.INV(1–B14,Mean,Stdev)

and

5NORM.INV(B15,Mean,Stdev)

To accomplish its objective, ZTel needs to raise the automatic rejection point from 425 to 455 and lower the automatic acceptance point from 600 to 582.

09953_ch05_ptg01_183-241.indd 210 04/03/19 12:25 PM

Figure 5.12 Calculations for Paper Quality Example

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

A B C D E F G H I J K Paper Machine Settings Range names used:

Mean =$B$3 $B$5=

Stdev_good Mean Stdev in good case

Stdev_bad =$B$4

Stdev in bad case

Reject region Lower limit Upper limit

Probability of reject in good case 0.024 =NORM.DIST(B8,Mean,Stdev_good,TRUE)+(1-NORM.DIST(B9,Mean,Stdev_good,TRUE)) in bad case 0.114 =NORM.DIST(B8,Mean,Stdev_bad,TRUE)+(1-NORM.DIST(B9,Mean,Stdev_bad,TRUE))

Data table of rejection probability as a function of the mean and good standard deviation Standard deviation

0.024 0.1 0.11 0.12 0.13 0.14 0.15 19.7 0.841 0.818 0.798 0.779 0.762 0.748 19.8 0.500 0.500 0.500 0.500 0.500 0.500 19.9 0.159 0.182 0.203 0.222 0.240 0.256

Mean 20 0.024 0.038 0.054 0.072 0.093 0.114 20.1 0.024 0.038 0.054 0.072 0.093 0.114 20.2 0.159 0.182 0.203 0.222 0.240 0.256 20.3 0.500 0.500 0.500 0.500 0.500 0.500 20.4 0.841 0.818 0.798 0.779 0.762 0.748

The point of this example is that the probability of a reject, i.e., falling outside the allowed limits, can vary greatly, not only when the mean is off target, but also when the standard deviation increases. The illustrates the famous line that “variability is the enemy.”

20 0.1

0.15

19.8 20.3

Figure 5.13 Rejection Regions for Paper Quality Example

5-4 The Normal Distribution 2 1 1

EXAMPLE

5.7 ANALYZING AN INVESTOR’S AFTER-TAX PROFIT Howard Davis invests $10,000 in a certain stock on January 1. By examining past movements of this stock and consulting with his broker, Howard estimates that the annual return from this stock, X, is normally distributed with mean 5% and stan- dard deviation 14%. Here X (when expressed as a decimal) is the profit Howard receives per dollar invested. It means that on December 31, his $10,000 will have grown to 10,000 11 1 X2 dollars. Because Howard is in the 33% tax bracket, he will then

09953_ch05_ptg01_183-241.indd 211 04/03/19 12:25 PM

have to pay the Internal Revenue Service 33% of his profit, if he makes a profit. However, he doesn’t have to pay any tax if he loses money. Calculate the probability that Howard will have to pay the IRS at least $400, and calculate the probability that he won’t have to pay any tax. Also, calculate the dollar amount such that Howard’s after-tax profit is 90% certain to be less than this amount; that is, calculate the 90th percentile of his after-tax profit.

Objective To determine the after-tax profit Howard Davis can be 90% certain of earning.

Solution Howard’s before-tax profit is 10,000X dollars, so the amount he pays the IRS is 0.33110,000X2, or 3300X dollars. We want the probability that this is at least $400. Because 3300X 7 400 is the same as X 7 4/33, the probability of this outcome can be found as in Figure 5.14. (See the file Tax on Stock Return Finished.xlsx.) It is calculated in cell B8 with the formula spelled out to its right. As you can see, Howard has about a 30% chance of paying at least $400 in taxes.

Figure 5.14 Calculations for Taxable Returns Example

1 2 3 4 5 6 7 8 9

10 11 12

IHGFEDCBA Tax on Stock Return

Range names used: Amount $B$3=detsevni_tnuomAdetsevni

$B$4=naeMnaeM $B$5=vedtSvedtS

Tax $B$6=etar_xaTetar

Probability Probability of no tax

he pays at least $400 in taxes 0.305 0.360

=1-NORM.DIST(400/(Amount_invested*Tax_rate),Mean,Stdev,TRUE) =NORM.DIST(0,Mean,Stdev,TRUE)

90th percentile of stock return 22.94% =NORM.INV(0.9,Mean,Stdev) 90th percentile of after-tax return $1,537 =(1-Tax_rate)*Amount_invested*B10

000,01$ %5 %14 %33

The probability that he doesn’t have to pay any tax is easier. It is the probability the return on the stock is negative. This is 0.36, found in cell B9 with the formula shown to its right.

To answer the last question, note that the after-tax profit (when X is positive) is 67% of the before-tax profit, or 6700X dollars, and we want its 90th percentile. If this percentile is x, then we know that P16700X 6 x2 5 0.90, which is the same as P1X 6 x>67002 5 0.90. In words, we want the 90th percentile of the X distribution to be x/6700. From cell B11 of Figure 5.14, the 90th percentile is 22.94%, so the required value of x is $1,537.

is approximately normal with mean $75 and standard deviation $20. a. What is the probability that a randomly selected cus-

tomer spends less than $85 at this store? b. What is the probability that a randomly selected cus-

tomer spends between $65 and $85 at this store? c. What is the probability that a randomly selected cus-

tomer spends more than $45 at this store? d. Find the dollar amount such that 75% of all customers

spend no more than this amount. e. Find the dollar amount such that 80% of all customers

spend at least this amount. f. Find two dollar amounts, equidistant from the mean,

such that 90% of all customer purchases are between these values.

15. A machine used to regulate the amount of a certain chemical dispensed in the production of a particular type of cough syrup can be set so that it discharges an aver- age of m milliliters (ml) of the chemical in each bottle

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 13. The grades on the midterm examination given in a large

managerial statistics class are normally distributed with mean 75 and standard deviation 9. The instructor of this class wants to assign an A grade to the top 10% of the scores, a B grade to the next 10% of the scores, a C grade to the next 10% of the scores, a D grade to the next 10% of the scores, and an F grade to all scores below the 60th per- centile of this distribution. For each possible letter grade, find the lowest acceptable score within the established range. For example, the lowest acceptable score for an A is the score at the 90th percentile of this normal distribution.

14. Suppose it is known that the distribution of purchase amounts by customers entering a popular retail store

2 1 2 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

09953_ch05_ptg01_183-241.indd 212 04/03/19 12:25 PM

5-4 The Normal Distribution 2 1 3

of cough syrup. The amount of chemical placed into each bottle of cough syrup is known to have a normal distribution with a standard deviation of 0.250 ml. If this machine discharges more than 2 ml of the chemi- cal when preparing a given bottle of this cough syrup, the bottle is considered to be unacceptable by industry standards. Determine the setting for m so that no more than 1% of the bottles of cough syrup prepared by this machine will be rejected.

16. Assume that the monthly sales for Toyota passenger cars follow a normal distribution with mean 5000 cars and standard deviation 1400 cars. a. There is a 1% chance that Toyota will sell more than

what number of passenger cars during the next year? (You can assume that sales in different months are probabilistically independent.)

b. What is the probability that Toyota will sell between 55,000 and 65,000 passenger cars during the next year?

17. An investor has invested in nine different investments. The dollar returns on the different investments are probabilistically independent, and each return follows a normal distribution with mean $50,000 and standard deviation $10,000. a. There is a 1% chance that the total return on the nine

investments is less than what value? b. What is the probability that the investor’s total return

is between $400,000 and $520,000? 18. Suppose that the weight of a typical American male

follows a normal distribution with m 5 180 lb and s 5 30 lb. Also, suppose 91.92% of all American males weigh more than I weigh. a. What fraction of American males weigh more than

225 pounds? b. How much do I weigh? c. If I weighed 20 pounds more than I do, what percen-

tile would I be in? 19. Assume that the length of a typical televised base-

ball game, including all the commercial timeouts, is normally distributed with mean 2.45 hours and standard deviation 0.37 hour. Consider a televised baseball game that begins at 2:00 in the afternoon. The next regularly scheduled broadcast is at 5:00. a. What is the probability that the game will cut into the

next show, that is, go past 5:00? b. If the game is over before 4:30, another half-hour

show can be inserted into the 4:30–5:00 slot. What is the probability of this occurring?

20. The amount of a soft drink that goes into a typical 12-ounce can varies from can to can. It is normally dis- tributed with an adjustable mean m and a fixed standard deviation of 0.05 ounce. (The adjustment is made to the filling machine.) a. If regulations require that cans have at least 11.9

ounces, what is the smallest mean m that can be used so that at least 99.5% of all cans meet the regulation?

b. If the mean setting from part a is used, what is the probability that a typical can has at least 12 ounces?

21. Suppose that the demands for a company’s product in weeks 1, 2, and 3 are each normally distributed. The means are 50, 45, and 65. The standard deviations are 10, 5, and 15. Assume that these three demands are probabilistically independent. a. Suppose that the company currently has 180 units in

stock, and it will not be receiving any more shipments from its supplier for at least three weeks. What is the probability that stock will run out during this three- week period?

b. How many units should the company currently have in stock so that it can be 98% certain of not running out during this three-week period? Again, assume that it won’t receive any more shipments during this period.

Level B 22. Matthew’s Bakery prepares peanut butter cookies for sale

every morning. It costs the bakery $0.50 to bake each peanut butter cookie, and each cookie is sold for $1.25. At the end of the day, leftover cookies are discounted and sold the following day at $0.40 per cookie. The daily demand (in dozens) for peanut butter cookies at this bak- ery is known to be normally distributed with mean 200 and standard deviation 60. The manager of Matthew’s Bakery is trying to determine how many dozen peanut butter cookies to make each morning to maximize the product’s contribution to bakery profits. Use simulation to find a very good, if not optimal, production plan.

23. The manufacturer of a particular bicycle model has the following costs associated with the management of this product’s inventory. In particular, the company currently maintains an inventory of 1000 units of this bicycle model at the beginning of each year. If X units are demanded each year and X is less than 1000, the excess supply, 1000 2 X units, must be stored until next year at a cost of $50 per unit. If X is greater than 1000 units, the excess demand, X 2 1000 units, must be produced separately at an extra cost of $80 per unit. Assume that the annual demand (X) for this bicycle model is normally distributed with mean 1000 and standard deviation 75. a. Find the expected annual cost associated with man-

aging potential shortages or surpluses of this product. (Hint: Use simulation to approximate the answer. An exact solution using probability arguments is beyond the level of this book.)

b. Find two annual total cost levels, equidistant from the expected value found in part a, such that 95% of all costs associated with managing potential shortages or surpluses of this product are between these values. (Continue to use simulation.)

c. Comment on this manufacturer’s annual production policy for this bicycle model in light of your findings in part b.

09953_ch05_ptg01_183-241.indd 213 04/03/19 12:25 PM

2 1 4 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

5-5 The Binomial Distribution The normal distribution is undoubtedly the most important probability distribution in statis- tics. Not far behind, however, is the binomial distribution. The binomial distribution is a discrete distribution that can occur in two situations: (1) when sampling from a population with only two types of members (males and females, for example), and (2) when perform- ing a sequence of identical experiments, each of which has only two possible outcomes.

24. It is widely known that many drivers on interstate highways in the United States do not observe the posted speed limit. Assume that the actual rates of speed driven by U.S. motor- ists are normally distributed with mean m mph and standard deviation 5 mph. Given this information, answer each of the following independent questions. (Hint: Use Goal Seek in parts a and b, and use the Solver add-in with no objective in part c. Solver is usually used to optimize, but it can also be used to solve equations with multiple unknowns.) a. If 40% of all U.S. drivers are observed traveling at 65

mph or more, what is the mean m? b. If 25% of all U.S. drivers are observed traveling at 50

mph or less, what is the mean m? c. Suppose now that the mean m and standard devia-

tion s of this distribution are both unknown. Further- more, it is observed that 40% of all U.S. drivers travel at less than 55 mph and 10% of all U.S. drivers travel at more than 70 mph. What must m and s be?

25. The lifetime of a certain manufacturer’s washing machine is normally distributed with mean 4 years. Only 15% of all these washing machines last at least 5 years. What is the standard deviation of the lifetime of a washing machine made by this manufacturer?

26. A fast-food restaurant sells hamburgers and chicken sand- wiches. On a typical weekday the demand for hamburgers is normally distributed with mean 313 and standard devi- ation 57; the demand for chicken sandwiches is normally distributed with mean 93 and standard deviation 22. a. How many hamburgers must the restaurant stock to be

98% sure of not running out on a given day? b. Answer part a for chicken sandwiches.

c. If the restaurant stocks 400 hamburgers and 150 chicken sandwiches for a given day, what is the prob- ability that it will run out of hamburgers or chicken sandwiches (or both) that day? Assume that the demand for hamburgers and the demand for chicken sandwiches are probabilistically independent.

d. Why is the independence assumption in part c prob- ably not realistic? Using a more realistic assumption, do you think the probability requested in part c would increase or decrease?

27. Referring to the box plots introduced in Chapter 2, the sides of the “box” are at the first and third quartiles, and the difference between these (the length of the box) is called the interquartile range (IQR). Some implementa- tions of box plots define a mild outlier as an observa- tion that is between 1.5 and 3 IQRs from the box, and an extreme outlier as an observation that is more than 3 IQRs from the box. a. If the data are normally distributed, what percent-

age of values will be mild outliers? What percent- age will be extreme outliers? Why don’t the answers depend on the mean and/or standard deviation of the distribution?

b. Check your answers in part a with simulation. Simu- late a large number of normal random numbers (you can choose any mean and standard deviation), and count the number of mild and extreme outliers with appropriate formulas. Do these match, at least approx- imately, your answers to part a?

What the Binomial Distribution Describes

Unlike the normal distribution, which can describe many types of random phenomena, the binomial distribution is relevant for a very common and specific situation: the number of successes in a fixed number of trials, where the trials are probabilistically independent and the probability of success remains constant across trials. Whenever this situation occurs, the binomial distribution is the relevant distribution for the number of successes.

Fundamental Insight

For the latter case, imagine any experiment that can be repeated many times under identical conditions. It is common to refer to each repetition of the experiment as a trial.

09953_ch05_ptg01_183-241.indd 214 04/03/19 12:25 PM

5-5 The Binomial Distribution 2 1 5

We assume that the outcomes of successive trials are probabilistically independent of one another and that each trial has only two possible outcomes. We label these two possibil- ities generically as success and failure. In any particular application the outcomes might be Democrat/Republican, defective/nondefective, went bankrupt/remained solvent, and so on. We label the probability of a success on each trial as p, and the probability of a failure as 1 2 p. We let n be the number of trials.

Consider a situation where there are n independent, identical trials, where the probability of a success on each trial is p and the probability of a failure is 1 2 p. Define X to be the random number of successes in the n trials. Then X has a bino- mial distribution with parameters n and p.

For example, the binomial distribution with parameters 100 and 0.3 is the distribution of the number of successes in 100 trials when the probability of success is 0.3 on each trial. A simple example that you can keep in mind throughout this section is the number of heads you would see if you flipped a coin n times. Assuming the coin is well balanced, the relevant distribution is binomial with parameters n and p 5 0.5. This coin-flipping example is often used to illustrate the binomial distribution because of its simplicity, but you will see that the binomial distribution also applies to many important business situations.

To understand how the binomial distribution works, consider the coin-flipping exam- ple with n 5 3. If X represents the number of heads in three flips of the coin, then the possible values of X are 0, 1, 2, and 3. You can find the probabilities of these values by con- sidering the eight possible outcomes of the three flips: (T,T,T), (T,T,H), (T,H,T), (H,T,T), (T,H,H), (H,T,H), (H,H,T), and (H,H,H). Because of symmetry (the well-balanced prop- erty of the coin), each of these eight possible outcomes must have the same probability, so each must have probability 1>8. Next, note that one of the outcomes has X 5 0, three outcomes have X 5 1, three outcomes have X 5 2, and one outcome has X 5 3. There- fore, the probability distribution of X is

P1X 5 02 5 1>8, P1X 5 12 5 3>8, P1X 5 22 5 3>8, P1X 5 32 5 1>8 This is a special case of the binomial distribution, with n 5 3 and p 5 0.5. In general, where n can be any positive integer and p can be any probability between 0 and 1, there is a rather complex formula for calculating P 1X 5 k2 for any integer k from 0 to n. Instead of presenting this formula, we will discuss how to calculate binomial probabilities in Excel. You do this with the BINOM.DIST function. The general form of this function is

5BINOM.DIST(k, n, p,cum)

The middle two arguments are the number of trials n and the probability of success p on each trial. The first parameter k is an integer number of successes that you specify. The last parameter, cum, is either TRUE or FALSE. It is TRUE (or 1) if you want the probability of less than or equal to k successes, and it is FALSE (or 0) if you want the probability of exactly k successes. We illustrate typical binomial calculations, including those involving the BINOM.INV function for percentile calculations, in Example 5.8.

Old Binomial Functions

Prior to Excel 2010, the binomial functions were BINOMDIST and CRITBI- NOM. They are exactly the same as the new BINOM.DIST and BINOM.INV functions. Microsoft kept these old functions for backward compatibility, so you can continue to use them if you prefer.

Excel Tip

09953_ch05_ptg01_183-241.indd 215 04/03/19 12:25 PM

2 1 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

With this in mind, the probabilities requested in (a) through (f) become:

a. P1X 5 582 b. P1X # 652 c. P1X 6 702 5 P1X # 692 d. P1X $ 592 5 1 2 P1X 6 592 5 1 2 P1X # 582 e. P1X 7 652 5 1 2 P1X # 652 f. P155 # X # 652 5 P1X # 652 2 P1X 6 552 5 P1X # 652 2 P1X # 542

Note how we have manipulated each of these so that it includes only terms of the form P1X 5 k2 or P1X # k2 for appropriate values of k. These are the types of probabilities that can be handled directly by the BINOM.DIST function. The answers appear in the range B7:B12, and the corresponding formulas are shown (as labels) to their right.

The probabilities requested in (g) through (i) involve failures rather than successes. But because each trial results in either a success or a failure, the number of failures is also binomially distributed, with parameters n and 1 2 p 5 0.4. So in rows 14

EXAMPLE

5.8 BINOMIAL CALCULATIONS IN EXCEL Suppose that 100 identical batteries are inserted in identical flashlights. Each flashlight takes a single battery. After eight hours of continuous use, a given battery is still operating with probability 0.6 or has failed with probability 0.4. Let X be the number of successes in these 100 trials, where a success means that the battery is still functioning. Find the probabilities of the following events: (a) exactly 58 successes, (b) no more than 65 successes, (c) less than 70 successes, (d) at least 59 successes, (e) greater than 65 successes, (f) between 55 and 65 successes (inclusive), (g) exactly 40 failures, (h) at least 35 failures, and (i) less than 42 failures. Then find the 95th percentile of the distribution of X.

Objective To use Excel’s BINOM.DIST and BINOM.INV functions for calculating binomial probabilities and percentiles.

Solution Figure 5.15 shows the solution to all of these problems. (See the file Binomial Calculations Finished.xlsx.) The probabilities requested in parts (a) through (f) all involve the number of successes X. The key to these is the wording of phrases such as “no more than,” “greater than,” and so on. In particular, you have to be careful to distinguish between probabilities such as P1X 6 k2 and P1X # k2. The latter includes the possibility of having X 5 k and the former does not.

Figure 5.15 Typical Binomial Calculations 1

HGFEDCBA Binomial Probability Calculations

Range names used: n =$B$3 p =$B$4

=BINOM.DIST(58,n,p,FALSE) =BINOM.DIST(65,n,p,TRUE) =BINOM.DIST(69,n,p,TRUE) =1-BINOM.DIST(58,n,p,TRUE) =1-BINOM.DIST(65,n,p,TRUE) =BINOM.DIST(65,n,p,TRUE)-BINOM.DIST(54,n,p,TRUE)

=BINOM.DIST(40,n,1-p,FALSE) =1-BINOM.DIST(34,n,1-p,TRUE) =BINOM.DIST(41,n,1-p,TRUE)

=BINOM.DIST(A20,n,p,TRUE) (Copy down)

=BINOM.INV(n,p,B27)

Number of Probability of success on each trial

ytilibaborPtnevE Exactly 58 2470.0sesseccus No more than 65 7968.0sesseccus Less than 70 2579.0sesseccus At least 59 5226.0sesseccus Greater than 65 3031.0sesseccus Between 55 and 65 successes (inclusive) 0.7386

Exactly 40 2180.0seruliaf At least 35 7968.0seruliaf Less than 42 5226.0seruliaf

Finding the 95th percentile (trial and error) Trial values CumProb

65 0.8697 66 0.9087 67 0.9385 68 0.9602 69 0.9752 70 0.9852

68 0.95

001slairt 0.6

09953_ch05_ptg01_183-241.indd 216 04/03/19 12:25 PM

through 16 the requested probabilities are calculated in exactly the same way, except that 1-p is substituted for p in the third argument of the BINOM.DIST function.

Finally, to calculate the 95th percentile of the distribution of X, you can proceed by trial and error. For each value k from 65 to 70, the probability P1X # k2 is calculated in column B with the BINOM.DIST function. As you can see, there is no value k such that P1X # k2 5 0.95 exactly. Specifically, P1X # 672 is slightly less than 0.95 and P1X # 682 is slightly greater than 0.95. Therefore, the meaning of the “95th percentile” is somewhat ambiguous. If you want the largest value k such that P1X # k2 # 0.95, then this k is 67. If instead you want the smallest value k such that P1X # k2 $ 0.95, then this value is 68. The latter interpretation is the one usually accepted for binomial percentiles.

In fact, Excel has another built-in function, BINOM.INV for finding this value of k. This function is illustrated in row 27 of Figure 5.15 in cell A27. This BINOM.INV function returns 68, the smallest value k such that P1X # k2 $ 0.95 for this binomial distribution.

5-5 The Binomial Distribution 2 1 7

5-5a Mean and Standard Deviation of the Binomial Distribution

It can be shown that the mean and standard deviation of a binomial distribution with parameters n and p are given by the following equations.

Mean and Standard Deviation of the Binomial Distribution

E1X2 5 np (5.14) Stdev1X2 5 !np11 2 p2 (5.15)

The formula for the mean is quite intuitive. For example, if you observe 100 trials, each with probability of success 0.6, your best guess for the number of successes is 10010.62 5 60. The standard deviation is less obvious but still very useful. It indicates how far the actual number of successes is likely to deviate from the mean. In this case the standard deviation is !10010.6210.42 5 4.90.

Fortunately, the empirical rules discussed in Chapter 2 also apply, at least approxi- mately, to the binomial distribution. That is, there is about a 95% chance that the actual number of successes will be within two standard deviations of the mean, and there is almost no chance that the number of successes will be more than three standard deviations from the mean. So for this example, it is very likely that the number of successes will be in the range of approximately 50 to 70, and it is very unlikely that there will be fewer than 45 or more than 75 successes.

This reasoning is extremely useful. It provides a rough estimate of the number of successes you are likely to observe. Suppose 1000 parts are sampled randomly from an assembly line and, based on historical performance, the percentage of parts with some type of defect is about 5%. Translated into a binomial model, each of the 1000 parts, independently of the others, has some type of defect with probability 0.05. Would it be surprising to see, say, 75 parts with a defect? The mean is 100010.052 5 50 and the stan- dard deviation is !100010.05210.952 5 6.89. Therefore, the number of parts with defects is approximately 95% certain to be within 50 { 216.892, or approximately from 36 to 64. Because 75 is slightly beyond three standard deviations from the mean, it is highly unlikely that there would be 75 (or more) defective parts.

5-5b The Binomial Distribution in the Context of Sampling The binomial distribution also applies to sampling from a population with two types of members. Let’s say these two types are men and women, although in applications they might be Democrats and Republicans, users of our product and nonusers, and so on. We assume that the population has N members, of whom NM are men and NW are women (where NM 1 NW 5 N). If you sample n of these randomly, you are typically interested in

09953_ch05_ptg01_183-241.indd 217 04/03/19 12:25 PM

2 1 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

the composition of the sample. You might expect the number of men in the sample to be binomially distributed with parameters n and p 5 NM>N, the fraction of men in the popu- lation. However, this depends on how the sampling is performed.

If sampling is done without replacement, each member of the population can be sam- pled only once. That is, once a person is sampled, his or her name is struck from the list and cannot be sampled again. If sampling is done with replacement, then it is possible, although maybe not likely, to select a given member of the population more than once. Most real-world sampling is performed without replacement. There is no point in obtaining infor- mation from the same person more than once. However, the binomial model applies only to sampling with replacement. Because the composition of the remaining population keeps changing as the sampling progresses, the binomial model provides only an approximation if sampling is done without replacement. If there is no replacement, the value of p, the pro- portion of men in this case, does not stay constant, a requirement of the binomial model. The appropriate distribution for sampling without replacement is called the hypergeometric distribution, a distribution we will not discuss here.5

If n is small relative to N, however, the binomial distribution is a very good approxi- mation to the hypergeometric distribution and can be used even if sampling is performed without replacement. A rule of thumb is that if n is no greater than 10% of N, that is, no more than 10% of the population is sampled, then the binomial model can be used safely. Of course, most national polls sample considerably less than 10% of the population. In fact, they often sample only about a thousand people from the hundreds of millions in the entire population. The bottom line is that in most real-world sampling contexts, the bino- mial model is perfectly adequate.

5-5c The Normal Approximation to the Binomial If you graph the binomial probabilities, you will see an interesting phenomenon—namely, the graph begins to look symmetric and bell-shaped when n is fairly large and p is not too close to 0 or 1. An example is illustrated in Figure 5.16 with the parameters n 5 30 and p 5 0.4. Generally, if np 7 5 and n11 2 p2 7 5, the binomial distribution can be approx- imated well by a normal distribution with mean np and standard deviation !np11 2 p2. Before Excel functions were available, this fact was used to find approximate binomial probabilities from normal tables. Although this is no longer necessary, it is still useful to know that a binomial distribution resembles a normal distribution for large n.

If n is large and p is not too close to 0 or l, the binomial distribution is bell-shaped and can be approxi- mated well by the normal distribution.

Continuity Correction

Because the normal distribution is continuous and the binomial distribution is discrete, the normal approximation to the binomial can be improved slight- ly with a continuity correction. If you want to approximate a binomial prob- ability such as P136 # X # 452, expand the interval by 0.5 on each end in the normal approximation. That is, approximate with the normal probability P135.5 # X # 45.52. Similarly, approximate binomial P1X # 452 with normal P1X # 45.52, or binomial P1X $ 362 with normal P1X $ 35.52. Admittedly, this continuity correction is mostly of historical interest. With Excel’s binomial functions, there is no need to resort to a normal approximation.

Excel Tip

One practical consequence of the normal approximation to the binomial is that the empirical rules can be applied. That is, when the binomial distribution is approximately symmetric and bell-shaped, there is about a 68% chance that the number of successes will

5 Excel has a function HYPGEOM.DIST for sampling without replacement that works much like the BINOM. DIST function. You can look it up under the Statistical category of Excel functions.

09953_ch05_ptg01_183-241.indd 218 04/03/19 12:25 PM

5-5 The Binomial Distribution 2 1 9

be within one standard deviation of the mean. Similarly, there is about a 95% chance that the number of successes will be within two standard deviations of the mean, and the num- ber of successes will almost surely be within three standard deviations of the mean. Here, the mean is np and the standard deviation is !np11 2 p2.

5-5d Binomial Distribution Examples The binomial distribution finds many applications in the business world and elsewhere. We discuss a few typical applications in this section.

Figure 5.16 Bell-shaped Binomial Distribution

Relationship Between Normal and Binomial Distributions

If you look at a graph of a binomial distribution when n is fairly large and p is not too close to 0 or 1, you will see that the distribution is bell-shaped. This is no accident. It can be proven mathematically that the normal distribution provides a very good approximation to the binomial under these conditions (n large, p not too close to 0 or 1). One implication is that the empirical rules from Chapter 2 apply very well to binomial distributions, using the mean and standard deviation in Equations (5.14) and (5.15). For example, there is about a 95% chance that the number of successes will be within two standard deviations of the mean.

Fundamental Insight

EXAMPLE

5.9 IS THIS MUTUAL FUND REALLY A WINNER? An investment broker at the Michaels & Dodson Company claims that he has found a real winner. He has tracked a mutual fund that has beaten a standard market index in 37 of the past 52 weeks. Could this be due to chance, or has he really found a winner?

Objective To determine the probability of a mutual fund outperforming a standard market index at least 37 out of 52 weeks.

09953_ch05_ptg01_183-241.indd 219 04/03/19 12:25 PM

Solution The broker is no doubt tracking a lot of mutual funds, and he is probably reporting only the best of these. Therefore, we will check whether the best of many mutual funds could do at least this well purely by chance. To do this, we first specify what we mean by “purely by chance.” This means that each week, a given fund has a fifty-fifty chance of beating the market index, independently of performance in other weeks. In other words, the number of weeks where a given fund outperforms the market index is binomially distributed with n 5 52 and p 5 0.5. With this in mind, cell B6 of Figure 5.17 shows the probability that a given fund does at least as well—beats the market index at least 37 out of 52 weeks—as the reported fund. (See the Beating the Market Finished.xlsx file.) Because P1X $ 372 5 1 2 P1X # 362, the relevant formula is 51–BINOM.DIST(B3−1,B4,0.5,TRUE)

Obviously, this probability, 0.00159, is quite small. A single fund isn’t likely to beat the market this often purely by chance.

Figure 5.17 Binomial Calculations for Investment Example

1 2 3 4 5

6 7 8

9 10 11 12 13 14 15 16 17 18

A B C D E F G Beating the market

Weeks beating market index Total number of weeks

Probability of doing at least this well by chance 0.00159 =1-BINOM.DIST(B3-1,B4,0.5,TRUE)

Number of mutual funds Probability of at least one doing at least this well 0.471 =1-BINOM.DIST(0,B8,B6,TRUE)

Two-way data table of the probability in B9 as a function of values in B3 and B8 Number of weeks beating the market index

0.471 36 37 38 39 40 Number of mutual funds 200 0.542 0.273 0.113 0.040 0.013

300 0.690 0.380 0.164 0.060 0.019 400 0.790 0.471 0.213 0.079 0.025 500 0.858 0.549 0.258 0.097 0.031 600 0.904 0.616 0.301 0.116 0.038

37 52

400

However, the probability that the best of many mutual funds does at least this well is much larger. To calculate this prob- ability, assume that 400 funds are being tracked, and let Y be the number of these that beat the market at least 37 of 52 weeks. Then Y is also binomially distributed, with parameters n 5 400 and p 5 0.00159, the probability calculated previously. To see whether any of the 400 funds beats the market at least 37 of 52 weeks, calculate P1Y $ 12 5 1 2 P1Y 5 02 in cell B9 with the formula

51–BINOM.DIST(0,B8,B7,TRUE)

(Can you see why the fourth argument could be TRUE or FALSE?) The resulting probability is nearly 0.5—that is, there is nearly a fifty-fifty chance that at least one of 400 funds will do as well as the reported fund. This certainly casts doubt on the broker’s claim that he has found a real winner. It is more likely that his star fund just got lucky and will perform no better than average in succeeding weeks.

To see how the probability in cell B9 depends on the level of success of the reported fund (the value in cell B3) and the number of mutual funds being tracked (in cell B8), a two-way data table has been created in the range B13:G18. (The formula in cell B13 is = B9, the row input cell is B3, and the column input cell is B8.) As you saw, beating the market 37 times out of 52 is no big deal with 400 funds, but beating it 40 times out of 52, even with 600 funds, is something worth reporting. The probability of this happening purely by chance is only 0.038, or less than 1 out of 25.

2 2 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

09953_ch05_ptg01_183-241.indd 220 04/03/19 12:25 PM

5-5 The Binomial Distribution 2 2 1

The next example requires a normal calculation to find a probability p, which is then used in a binomial calculation.

EXAMPLE

5.10 DAILY SALES AT A SUPERMARKET Customers at a supermarket spend varying amounts. Historical data indicate that the amount spent per customer is normally distributed with mean $85 and standard deviation $30. If 500 customers shop in a given day, calculate the mean and standard deviation of the number who spend at least $100. Then calculate the probability that at least 30% of all customers spend at least $100.

Objective To use the normal and binomial distributions to calculate the typical number of customers who spend at least $100 per day and the probability that at least 30% of all 500 daily customers spend at least $100.

Solution Both questions involve the number of customers who spend at least $100. Because the amounts spent are normally distributed, the probability that a typical customer spends at least $100 is found with the NORM.DIST function. This probability, 0.309, appears in cell B7 of Figure 5.18. (See the file Supermarket Spending Finished.xlsx.) It is calculated with the formula

51-NORM.DIST(100,B4,B5,TRUE)

This probability is then used as the parameter p in a binomial model. The mean and standard deviation of the number who spend at least $100 are calculated in cells B13 and B14 as np and !np11 2 p2 using n 5 500, the number of shoppers, and p 5 0.309. The expected number who spend at least $100 is slightly greater than 154, and the standard deviation of this num- ber is slightly greater than 10.

To answer the second question, note that 30% of 500 customers is 150 customers. Then the probability that at least 30% of the customers spend at least $100 is the probability that a binomially distributed random variable, with n 5 500 and p 5 0.309, is at least 150. This binomial probability, which turns out to be about 2>3, is calculated in cell B16 with the formula 51-BINOM.DIST(0.3*B10-1,B10,B7,TRUE)

Note that the first argument evaluates to 149. This is because the probability of at least 150 customers is one minus the probability of less than or equal to 149 customers.

FEDCBA

Supermarket spending

Amount spent per customer (normally distributed) Mean StDev

Probability that a customer spends at least $100 0.309 =1-NORM.DIST(100,B4,B5,TRUE)

Number of customers

Mean and stdev of number who spend at least $100 =B10*B7154.27Mean =SQRT(B10*B7*(1-B7))10.33StDev

Probability at least 30% spend at least $100 0.676 =1-BINOM.DIST(0.3*B10-1,B10,B7,TRUE)

$85 $30

005

Figure 5.18 Calculations for Supermarket Example

09953_ch05_ptg01_183-241.indd 221 04/03/19 12:25 PM

2 2 2 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

EXAMPLE

5.11 OVERBOOKING BY AIRLINES This example presents a simplified version of calculations used by airlines when they overbook flights. They realize that a certain percentage of ticketed passengers will cancel at the last minute. Therefore, to avoid empty seats, they sell more tickets than there are seats, hoping that just about the right number of passengers show up. We assume that the no-show rate is 10%. In binomial terms, we assume that each ticketed passenger, independently of the others, shows up with probability 0.90 and cancels with probability 0.10.

For a flight with 200 seats, the airline wants to see how sensitive various probabilities are to the number of tickets it issues. In particular, it wants to calculate (a) the probability that more than 205 passengers show up, (b) the probability that more than 200 passengers show up, (c) the probability that at least 195 seats are filled, and (d) the probability that at least 190 seats are filled. The first two of these are “bad” events from the airline’s perspective; they mean that some customers will be bumped from the flight. The last two events are “good” in the sense that the airline wants most of the seats to be occupied.

Objective To assess the benefits and drawbacks of airline overbooking.

Solution To solve the airline’s problem, we use the BINOM.DIST function and a data table. The solution appears in Figure 5.19. (See the file Airline Overbooking Finished.xlsx.) For any number of tickets issued in cell B6, the required probabilities are calcu- lated in row 10. For example, the formulas in cells B10 and D10 are

51-BINOM.DIST(205,NTickets,1-PNoShow,TRUE)

Figure 5.19 Binomial Calculations for Overbooking Example

1 2 3 4 5 6 7 8

9 10 11 12

13 14 15 16 17 18 19 20 21 22 23 24

A B C D E Airline overbooking Range names used:

NTickets =$B$6 Number of seats PNoShow =$B$4 Probability of no-show

Number of tickets issued

Required probabilities More than 200

show up More than

205 show up At least 190

seats filled At least 195

seats filled

0.001 0.050 0.421 0.820

Data table showing sensitivity of probabilities to number of tickets issued

Number of tickets issued More than

205 show up More than 200

show up At least 195

seats filled At least 190

seats filled 0.001 0.050 0.421 0.820

0.384

0.370 0.607

206 0.000 0.000 0.012 0.171 209 0.000 0.001 0.064 212 0.000 0.009 0.201 0.628 215 0.001 0.050 0.421 0.820 218 0.013 0.166 0.659 0.931 221 0.064 224 0.194 227 0.406 0.802 0.981 0.999 230 0.639 0.920 0.995 1.000 233 0.822 0.974 0.999 1.000

0.9950.939 0.9780.839

002 0.1

215

09953_ch05_ptg01_183-241.indd 222 04/03/19 12:25 PM

5-5 The Binomial Distribution 2 2 3

The following example concerns a real problem that occurs every time you watch election returns on TV. This problem is of particular interest in light of the highly unusual events that took place during election night television coverage of the U.S. presidential election in 2000, where the networks declared Al Gore an early winner in at least one state that he eventually lost. The basic question is how soon the networks can declare one of the candidates the winner, based on early voting returns. Our example is somewhat unrealistic because it ignores the possibility that early tabulations can be biased one way or the other. For example, the earliest reporting precincts might be known to be more heavily in favor of the Democrat than the population in general. Nevertheless, the example indicates, at least approximately, why the networks are able to make early conclusions based on such seemingly small amounts of data.

and

51-BINOM.DIST(194,NTickets,1-PNoShow,TRUE)

Note that the condition “more than” requires a slightly different calculation from “at least.” The probability of more than 205 is one minus the probability of less than or equal to 205, whereas the probability of at least 195 is one minus the probabil- ity of less than or equal to 194. Also, note that a passenger who shows up is called a success. Therefore, the third argument of each BINOM.DIST function is one minus the no-show probability.

To see how sensitive these probabilities are to the number of tickets issued, a one-way data table was created at the bottom of the spreadsheet. It is one-way because there is only one input, the number of tickets issued, even though four output prob- abilities are tabulated. (To create the data table, list several possible numbers of tickets issued along the side in column A and create links in row 14 to the probabilities in row 10. That is, enter the formula = B10 in cell B14 and copy it across row 14. Then form a data table using the range A14:E24, no row input cell, and column input cell B6.)

The results are as expected. As the airline issues more tickets, there is a larger chance of having to bump passengers from the flight, but there is also a larger chance of filling most seats. In reality, the airline has to make a trade-off between these two, taking its various costs and prices into account.

EXAMPLE

5.12 PROJECTING ELECTION WINNERS FROM EARLY RETURNS We assume that there are N voters in the population, of whom NR will vote for the Republican and ND will vote for the Dem- ocrat. The eventual winner will be the Republican if NR 7 ND and will be the Democrat otherwise, but we won’t know which until all of the votes are tabulated. (To simplify the example, we assume there are only two candidates and that the election will not end in a tie.) Let’s suppose that a small percentage of the votes have been counted and the Republican is currently ahead 540 to 460. On what basis can the networks declare the Republican the winner, especially if there are millions of voters in the population?

Objective To use a binomial model to determine whether early returns reflect the eventual winner of an election between two candidates.

Solution Let n 5 1000 be the total number of votes that have been tabulated. If X is the number of Republican votes so far, we are given that X 5 540. Now we pose the following question. If the Democrat were going to be the eventual winner, that is, ND 7 NR, and we randomly sampled 1000 voters from the population, how likely is it that at least 540 of these voters would be in favor of the Republican? If this is very unlikely, then the only reasonable conclusion is that the Democrat will not be the eventual winner. This is the reasoning the networks might use to declare the Republican the winner so early in the tabulation.

We use a binomial model to see how unlikely the event “at least 540 out of 1000 ” is, assuming that the Democrat will be the eventual winner. We need a value for p, the probability that a typical vote is for the Republican. This probability should be

09953_ch05_ptg01_183-241.indd 223 04/03/19 12:25 PM

2 2 4 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

the proportion of voters in the entire population who favor the Republican. All we know is that this probability is less than 0.5, because we have assumed that the Democrat will eventually win. In Figure 5.20, we show how the probability of at least 540 out of 1000 varies with values of p less than, but close to, 0.5. (See the file Election Returns Finished.xlsx.)

We enter a trial value of 0.49 for p in cell B3 and then calculate the required probability in cell B9 with the formula

=1-BINOM.DIST(B6-1,B5,B3,TRUE)

Then we use this to create the data table at the bottom of the spreadsheet. This data table tabulates the probability of the given lead (at least 540 out of 1000) for various values of p less than 0.5. As shown in the last few rows, even if the eventual outcome were going to be a virtual tie—with the Democrat slightly ahead—there would still be very little chance of the Republican being at least 80 votes ahead so far. But because the Republican is currently ahead by 80 votes, the networks feel safe in declaring the Republican the winner. Admittedly, the probability model they use is more complex than our simple binomial model, but the idea is the same.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 28. Many vehicles used in space travel are constructed with

redundant systems to protect flight crews and their valu- able equipment. In other words, backup systems are included within many vehicle components so that if one or more systems fail, backup systems will assure the safe operation of the given component and thus the entire vehicle. For example, consider one particular com- ponent of the U.S. space shuttle that has n duplicated systems (i.e., one original system and n 2 1 backup sys- tems). Each of these systems functions, independently of the others, with probability 0.98. This shuttle component functions successfully provided that at least one of the n systems functions properly.

a. Find the probability that this shuttle component func- tions successfully if n 5 2.

b. Find the probability that this shuttle component functions successfully if n 5 4.

c. What is the minimum number n of duplicated systems that must be incorporated into this shuttle component to ensure at least a 0.9999 probability of successful operation?

29. Suppose that a popular hotel for vacationers in Orlando, Florida, has a total of 300 identical rooms. As many major airline companies do, this hotel has adopted an overbooking policy in an effort to maximize the usage of its available lodging capacity. Assume that each potential hotel customer holding a room reservation, independently of other customers, cancels the reserva- tion or simply does not show up at the hotel on a given night with probability 0.15.

Figure 5.20 Binomial Calculations for Voting Example

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18

FEDCBA Election returns

Population proportion for Republican

Votes tabulated so far Votes for Republican so far

P(at least this many R votes) - binomial 0.0009 =1-BINOM.DIST(B6-1,B5,B3,TRUE)

Data table showing sensitivity of this probability to population proportion for Republican Population proportion for Republican

0.0009 0.490 0.0009 0.492 0.0013 0.494 0.0020 0.496 0.0030 0.498 0.0043 0.499 0.0052

Probability

0.49

1000 540

09953_ch05_ptg01_183-241.indd 224 04/03/19 12:25 PM

5-5 The Binomial Distribution 2 2 5

a. Find the largest number of room reservations that this hotel can book and still be at least 95% sure that everyone who shows up at the hotel will have a room on a given night.

b. Given that the hotel books the number of reservations found in part a, find the probability that at least 90% of the available rooms will be occupied on a given night.

c. Given that the hotel books the number of reservations found in part a, find the probability that at most 80% of the available rooms will be occupied on a given night.

d. How does your answer to part a change as the required assurance rate increases from 95% to 97%? How does your answer to part a change as the required assurance rate increases from 95% to 99%?

e. How does your answer to part a change as the cancel- lation rate varies between 5% and 25% (in increments of 5%)? Assume now that the required assurance rate remains at 95%.

30. A production process manufactures items with weights that are normally distributed with mean 15 pounds and standard deviation 0.1 pound. An item is considered to be defective if its weight is less than 14.8 pounds or greater than 15.2 pounds. Suppose that these items are currently produced in batches of 1000 units. a. Find the probability that at most 5% of the items in a

given batch will be defective. b. Find the probability that at least 90% of the items in a

given batch will be acceptable. c. How many items would have to be produced in a

batch to guarantee that a batch consists of no more than 1% defective items?

31. Past experience indicates that 30% of all individu- als entering a certain store decide to make a purchase. Using (a) the binomial distribution and (b) the normal approximation to the binomial, find that probability that 10 or more of the 30 individuals entering the store in a given hour will decide to make a purchase. Compare the results obtained using the two different approaches. Under what conditions will the normal approximation to this binomial probability become even more accurate?

32. Suppose that the number of ounces of soda put into a soft-drink can is normally distributed with m 5 12.05 ounces and s 5 0.03 ounce. a. Legally, a can must contain at least 12 ounces of soda.

What fraction of cans will contain at least 12 ounces of soda?

b. What fraction of cans will contain less than 11.9 ounces of soda?

c. What fraction of cans will contain between 12 and 12.08 ounces of soda?

d. One percent of all cans will weigh more than what value?

e. Ten percent of all cans will weigh less than what value?

f. The soft-drink company controls the mean weight in a can by setting a timer. For what mean should the timer be set so that only 1 in 1000 cans will be underweight?

g. Every day the company produces 10,000 cans. The government inspects 10 randomly chosen cans each day. If at least two are underweight, the company is fined $10,000. Given that m 5 12.05 ounces and s 5 0.03 ounce, what is the probability that the com- pany will be fined on a given day?

33. Suppose that 53% of all registered voters prefer presi- dential candidate Smith to presidential candidate Jones. (You can substitute the names of the most recent presi- dential candidates.) a. In a random sample of 100 voters, what is the proba-

bility that the sample will indicate that Smith will win the election (that is, there will be more votes in the sample for Smith)?

b. In a random sample of 100 voters, what is the proba- bility that the sample will indicate that Jones will win the election?

c. In a random sample of 100 voters, what is the prob- ability that the sample will indicate a dead heat (fifty-fifty)?

d. In a random sample of 100 voters, what is the prob- ability that between 40 and 60 (inclusive) voters will prefer Smith?

34. Assume that, on average, 95% of all ticket holders show up for a flight. If a plane seats 200 people, how many tickets should be sold to make the chance of an over- booked flight as close as possible to 5%?

35. Suppose that 55% of all people prefer Coke to Pepsi. We randomly choose 500 people and ask them if they pre- fer Coke to Pepsi. What is the probability that our sur- vey will (erroneously) indicate that Pepsi is preferred by more people than Coke? Does this probability increase or decrease as we take larger and larger samples? Why?

36. Suppose that 4% of all tax returns are audited. In a group of n tax returns, consider the probability that at most two returns are audited. How large must n be before this probability is less than 0.01?

37. Suppose that the height of a typical American female is normally distributed with m 5 64 inches and s 5 4 inches. We observe the height of 500 American females. a. What is the probability that fewer than 35 of the 500

women will be less than 58 inches tall? b. Let X be the number of the 500 women who are less

than 58 inches tall. Find the mean and standard devia- tion of X.

Level B 38. Many firms utilize sampling plans to control the quality

of manufactured items ready for shipment. To illustrate the use of a sampling plan, suppose that a particular com- pany produces and ships electronic computer chips in lots, each lot consisting of 1000 chips. This company’s

09953_ch05_ptg01_183-241.indd 225 04/03/19 12:25 PM

2 2 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

sampling plan specifies that quality control personnel should randomly sample 50 chips from each lot and accept the lot for shipping if the number of defective chips is four or fewer. The lot will be rejected if the num- ber of defective chips is five or more. a. Find the probability of accepting a lot as a func-

tion of the actual fraction of defective chips. In particular, let the actual fraction of defective chips in a given lot equal any of 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, 0.14, 0.16, 0.18. Then compute the lot acceptance probability for each of these lot defec- tive fractions.

b. Construct a graph showing the probability of lot acceptance for each of the lot defective fractions, and interpret your graph.

c. Repeat parts a and b under a revised sampling plan that calls for accepting a given lot if the number of defective chips found in the random sample of 50 chips is five or fewer. Summarize any notable differ- ences between the two graphs.

39. A standardized test consists entirely of multiple-choice questions, each with five possible choices. You want to ensure that a student who randomly guesses on each question will obtain an expected score of zero. How can you accomplish this?

40. In the current tax year, suppose that 5% of the millions of individual tax returns are fraudulent. That is, they contain errors that were purposely made to cheat the government. a. Although these errors are often well concealed, let’s

suppose that a thorough IRS audit will uncover them. If a random 250 tax returns are audited, what is the probability that the IRS will uncover at least 15 fraud- ulent returns?

b. Answer the same question as in part a, but this time assume there is only a 90% chance that a given fraud- ulent return will be spotted as such if it is audited.

41. Suppose you work for a survey research company. In a typical survey, you mail questionnaires to 150 companies. Of course, some of these companies might decide not to respond. Assume that the nonresponse rate is 45%; that is, each company’s probability of not responding, inde- pendently of the others, is 0.45. a. If your company requires at least 90 responses for a

valid survey, find the probability that it will get this

many. Use a data table to see how your answer varies as a function of the nonresponse rate (for a reasonable range of response rates surrounding 45%).

b. Suppose your company does this survey in two “waves.” It mails the 150 questionnaires and waits a certain period for the responses. As before, assume that the nonresponse rate is 45%. However, after this initial period, your company follows up (by tele- phone, say) on the nonrespondents, asking them to please respond. Suppose that the nonresponse rate on this second wave is 70%; that is, each original non- respondent now responds with probability 0.3, inde- pendently of the others. Your company now wants to find the probability of obtaining at least 110 responses total. It turns out that this is a difficult probability to calculate directly. So instead, approximate it with simulation.

42. Suppose you are sampling from a large population, and you ask the respondents whether they believe men should be allowed to take paid paternity leave from their jobs when they have a new child. Each person you sam- ple is equally likely to be male or female. The popula- tion proportion of females who believe males should be granted paid paternity leave is 56%, and the population proportion of males who favor it is 48%. If you sam- ple 200 people and count the number who believe males should be granted paternity leave, is this number bino- mially distributed? Explain why or why not. Would your answer change if you knew your sample was going to consist of exactly 100 males and 100 females?

43. A woman claims that she is a fortune-teller. Specif- ically, she claims that she can predict the direction of the change (up or down) in the Dow Jones Industrial Average for the next 10 days. For example, one possible prediction might be U, U, D, U, D, U, U, D, D, D. (You can assume that she makes all 10 predictions right now, although that does not affect your answer to the ques- tion.) Obviously, you are skeptical, thinking that she is just guessing, so you would be surprised if her pre- dictions are accurate. Which would surprise you more: (1) she predicts at least 8 out of 10 correctly, or (2) she predicts at least 6 out of 10 correctly on each of four separate occasions? Answer by assuming that (1) she is really guessing and (2) each day the Dow is equally likely to go up or down.

5-6 The Poisson and Exponential Distributions The final two distributions in this chapter are called the Poisson and exponential distributions. In most statistical applications, including those in the rest of this book, these distributions play a much less important role than the normal and binomial distributions. For this reason, we will not analyze them in much detail. However, in many applied man- agement science models, the Poisson and exponential distributions are key distributions.

09953_ch05_ptg01_183-241.indd 226 04/03/19 12:25 PM

5-6 The Poisson and Exponential Distributions 2 2 7

For example, much of the study of probabilistic inventory models, queueing models, and reliability models relies heavily on these two distributions.

5-6a The Poisson Distribution The Poisson distribution is a discrete distribution. It usually applies to the number of events occurring within a specified period of time or space. Its possible values are all of the nonnegative integers: 0, 1, 2, and so on—there is no upper limit. Even though there is an infinite number of possible values, this causes no real problems because the probabili- ties of all sufficiently large values are essentially 0.

The Poisson distribution is characterized by a single parameter, usually labeled l (Greek lambda), which must be positive. By adjusting the value of l, we are able to produce different Poisson distributions, all of which have the same basic shape as in Figure 5.21. That is, they first increase and then decrease. It turns out that l is easy to interpret. It is both the mean and the variance of the Poisson distribution. Therefore, the standard deviation is !l.

Figure 5.21 Typical Poisson Distribution

Typical Examples of the Poisson Distribution

• A bank manager is studying the arrival pattern to the bank. The events are customer arrivals, the number of arrivals in an hour is Poisson distributed, and l represents the expected number of arrivals per hour.

• An engineer is interested in the lifetime of a type of battery. A device that uses this type of battery is operated continuously. When the first battery fails, it is replaced by a sec- ond; when the second fails, it is replaced by a third, and so on. The events are battery failures, the number of failures that occur in a month is Poisson distributed, and l rep- resents the expected number of failures per month.

• A retailer is interested in the number of customers who order a particular product in a week. Then the events are customer orders for the product, the number of customer orders in a week is Poisson distributed, and l is the expected number of orders per week.

• In a quality control setting, the Poisson distribution is often relevant for describing the number of defects in some unit of space. For example, when paint is applied to the body of a new car, any minor blemish is considered a defect. Then the number of defects on the hood, say, might be Poisson distributed. In this case, l is the expected number of defects per hood.

These examples are representative of the many situations where the Poisson distribution has been applied. The parameter l is often called a rate—arrivals per hour, failures per month, and so on. If the unit of time (or space) is changed, the rate must be modified

09953_ch05_ptg01_183-241.indd 227 04/03/19 12:25 PM

2 2 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

accordingly. For example, if the number of arrivals to a bank in a single hour is Poisson distributed with rate l 5 30, then the number of arrivals in a half-hour period is Poisson distributed with rate l 5 15.

You can use Excel to calculate Poisson probabilities much as you did with binomial probabilities. The relevant function is the POISSON function. It takes the form

=POISSON.DIST (k,l,cum)

The third argument cum works exactly as in the binomial case. If it is FALSE (or 0), the function returns P1X 5 k2; if it is TRUE (or 1), the function returns P1X # k2. As examples, if l 5 5 5POISSON.DIST17,5,FALSE2 returns the probability of exactly 7, 5POISSON.DIST17,5,TRUE2 returns the probability of less than or equal to 7, and 5 1-POISSON.DIST13,5,TRUE2 returns the probability of greater than 3.

The next example shows how a manager or consultant could use the Poisson distribution.

EXAMPLE

5.13 MANAGING TV INVENTORY AT KRIEGLAND Kriegland is a department store that sells various brands of plasma screen TVs. One of the manager’s biggest problems is to decide on an appropriate inventory policy for stocking TVs. He wants to have enough in stock so that customers receive their requests right away, but he does not want to tie up too much money in inventory that sits on the storeroom floor.

Most of the difficulty results from the unpredictability of customer demand. If this demand were constant and known, the manager could decide on an appropriate inventory policy fairly easily. But the demand varies widely from month to month in a random manner. All the manager knows is that the historical average demand per month is approximately 17. Therefore, he decides to call in a consultant. The consultant immediately suggests using a probability model. Specifically, she attempts to find the probability distribution of demand in a typical month. How might she proceed?

Objective To model the probability distribution of monthly demand for plasma screen TVs with a particular Poisson distribution.

Solution Let X be the demand in a typical month. The consultant knows that there are many possible values of X. For example, if his- torical records show that monthly demands have always been between 0 and 40, the consultant knows that almost all of the probability should be assigned to the values 0 through 40. However, she does not relish the thought of finding 41 probabilities, P1X 5 02 through P1X 5 402, that sum to 1 and reflect historical frequencies. Instead, she discovers from the manager that the histogram of demands from previous months is shaped much like the graph in Figure 5.21. That is, it rises to some peak and then falls.

Knowing that a Poisson distribution has this same basic shape, the consultant decides to model the monthly demand with a Poisson distribution. To choose a particular Poisson distribution, all she has to do is choose a value of l, the mean demand per month. Because the historical average is approximately 17, she chooses l 5 17. Now she can test the Poisson model by calculating probabilities of various events and asking the manager whether these probabilities are reasonable approximations to reality.

For example, the Poisson probability that monthly demand is less than or equal to 20, P1X # 202, is 0.805 [using the Excel function POISSON.DIST120,17,TRUE2], and the probability that demand is between 10 and 15 inclusive, P110 # X # 152, is 0.345 [using POISSON.DIST115,17,TRUE2-POISSON.DIST19,17,TRUE2]. Figure 5.22 illustrates various probability calculations and shows the graph of the individual Poisson probabilities. (See the file Poisson Demand Distribution Finished.xlsx.)

09953_ch05_ptg01_183-241.indd 228 04/03/19 12:25 PM

5-6 The Poisson and Exponential Distributions 2 2 9

If the manager believes that these probabilities and other similar probabilities are reasonable, then the statistical part of the consultant’s job is finished. Otherwise, she must try a different Poisson distribution—a different value of l—or perhaps a different type of distribution altogether.

Value

8 0 00719

8 0 007

17 0 096

1 JIHGFEDCBA

Poisson distribution for monthly demand 2 3 4 5 6 7 8 9

Range name used: Mean monthly demand Mean =$B$3

Representative probability calculations Less than or equal to 20 0.805 =POISSON.DIST(20,Mean,TRUE) Between 10 and 15 (inclusive) 0.345 =POISSON.DIST(15,Mean,TRUE)-POISSON.DIST(9,Mean,TRUE)

Individual probabilities Value Probability

11 12 13 14 15 16 17 18

0 0.000 =POISSON.DIST(A11,Mean,FALSE)

7 0.003

20 21 22 23 24 25 26 27

10 0.023

28 29 30 31

1 0.000 copy down 2 0.000 3 0.000 4 0.000 5 0.000 6 0.001

9 0.014

11 0.036 12 0.050 13 0.066 14 0.080 15 0.091 16 0.096 17 0.096 18 0.091 19 0.081 20 0.069

Poisson Distribution with Mean 17 0.120

0.100

0.080

0.060

0.040

0.020

0.000 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

Figure 5.22 Poisson Calculations for TV Example

5-6b The Exponential Distribution Suppose that a bank manager is studying the pattern of customer arrivals at her branch location. As indicated previously in this section, the number of arrivals in an hour at a facility such as a bank is often well described by a Poisson distribution with parameter l, where l represents the expected number of arrivals per hour. An alternative way to view the uncertainty in the arrival process is to consider the times between customer arrivals. The most common probability distribution used to model these times, often called interar- rival times, is the exponential distribution.

In general, the continuous random variable X has an exponential distribution with parameter l (with l 7 0) if the density function of X has the form f1x2 5 le 2 lx for x 7 0. This density function has the shape shown in Figure 5.23. Because this density function decreases continuously from left to right, its most likely value is x 5 0. Alternatively, if you collect many observations from an exponential distribution and draw a histogram of the observed values, you should expect it to resemble the smooth curve shown in Figure 5.23, with the tallest bars to the left. The mean and standard deviation of this dis- tribution are easy to remember. They are both equal to the reciprocal of the parameter l. For example, an exponential distribution with parameter l 5 0.1 has mean and standard deviation both equal to 10.

09953_ch05_ptg01_183-241.indd 229 04/03/19 12:25 PM

2 3 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

As with the normal distribution, you usually want probabilities to the left or right of a given value. For any exponential distribution, the probability to the left of a given value x 7 0 can be calculated with Excel’s EXPON.DIST function. This function takes the form

=EXPON.DIST(x,l,TRUE)

For example, if x 5 0.5 and l 5 5 (so that the mean equals 1>5 5 0.2), the probabil- ity of being less than 0.5 can be found with the formula

=EXPON.DIST(0.5,5,TRUE)

This returns the probability 0.918. Of course, the probability of being greater than 0.5 is then 1 2 0.918 5 0.082.

Returning to the bank manager’s analysis of customer arrival data, when the times between arrivals are exponentially distributed, you sometimes hear that “arrivals occur according to a Poisson process.” This is because there is a close relationship between the exponential distribution, which measures times between events such as arrivals, and the Poisson distribution, which counts the number of events in a certain length of time. The details of this relationship are beyond the level of this book, so we will not explore the topic further. But if you hear, for example, that customers arrive at a facility accord- ing to a Poisson process at the rate of six per hour, then the corresponding times between arrivals are exponentially distributed with mean 1/6 hour.

Figure 5.23 Exponential Density Function

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 44. The annual number of industrial accidents occurring in

a particular manufacturing plant is known to follow a Poisson distribution with mean 12. a. What is the probability of observing exactly 12 acci-

dents during the coming year? b. What is the probability of observing no more than 12

accidents during the coming year? c. What is the probability of observing at least 15 acci-

dents during the coming year? d. What is the probability of observing between 10 and

15 accidents (inclusive) during the coming year?

e. Find the smallest integer k such that we can be at least 99% sure that the annual number of accidents occur- ring will be less than k.

45. Suppose the number of baskets scored by the Indiana University basketball team in one minute follows a Poisson distribution with l 5 1.5. In a 10-minute span of time, what is the probability that Indiana University scores exactly 20 baskets; at most 20 baskets? (Use the fact that if the rate per minute is l, then the rate in t minutes is lt.)

46. Suppose that the times between arrivals at a bank during the peak period of the day are exponentially distrib- uted with a mean of 45 seconds. If you just observed an arrival, what is the probability that you will need to wait for more than a minute before observing the next arrival? What is the probability you will need to wait at least two minutes?

09953_ch05_ptg01_183-241.indd 230 04/03/19 12:25 PM

5-7 Conclusion 2 3 1

Level B 47. Consider a Poisson random variable X with parameter

l 5 2. a. Find the probability that X is within one standard

deviation of its mean. b. Find the probability that X is within two standard

deviations of its mean. c. Find the probability that X is within three standard

deviations of its mean. d. Do the empirical rules we learned previously seem to

be applicable in working with the Poisson distribution where l 5 2? Explain why or why not.

e. Repeat parts a through d for the case of a Poisson ran- dom variable where l 5 20.

48. Based on historical data, the probability that a major league pitcher pitches a no-hitter in a game is about 1/1300. a. Use the binomial distribution to determine the proba-

bility that in 650 games 0, 1, 2, or 3 no-hitters will be pitched. (Find the separate probabilities of these four events.)

b. Repeat part a using the Poisson approximation to the binomial. This approximation says that if n is large and p is small, a binomial distribution with parame- ters n and p is approximately the same as a Poisson distribution with l 5 np.

5-7 Conclusion We have covered a lot of ground in this chapter, and much of the material, especially that on the normal distribution, will be used in later chapters. The normal distribution is the cornerstone for much of statistical theory. As you will see in later chapters on statistical inference and regression, an assumption of normality is behind most of the procedures we use. Therefore, it is important for you to understand the properties of the normal distribution and how to work with it in Excel. The binomial, Pois- son, and exponential distributions, although not used as frequently as the normal distribution in this book, are also extremely important. The examples we have discussed indicate how these distributions can be used in a variety of business situations. Finally, whenever a situation involves uncertainty, the rules of probability are relevant. Although we haven’t discussed the math- ematical details of probability in much depth, you should at least be familiar with the basic rules of probability discussed here.

Summary of Key Terms TERM EXPLANATION EXCEL PAGE EQUATION

Random variable Associates a numeric value with each possible outcome in a situation involving uncertainty

140

Probability A number between 0 and 1 that measures the likeli- hood that some event will occur

142

Rule of complements The probability of any event and the probability of its complement sum to 1

Basic formulas

142 5.1

Mutually exclusive events Events where only one of them can occur 142

Exhaustive events Events where at least one of them must occur 143

Addition rule for mutually exclusive events

The probability that at least one of a set of mutually exclusive events will occur is the sum of their probabilities

Basic formulas

143 5.2

Conditional probability formula

Updates the probability of an event, given the knowl- edge that another event has occurred

Basic formulas

144 5.3

Multiplication rule Formula for the probability that two events both occur Basic formulas

144 5.4

Probability tree A graphical representation of how events occur through time, useful for calculating probabilities of multiple events

145

Probabilistically indepen- dent events

Events where knowledge that one of them has occurred is of no value in assessing the probability that the other will occur

146 5.5

09953_ch05_ptg01_183-241.indd 231 04/03/19 12:25 PM

2 3 2 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

TERM EXPLANATION EXCEL PAGE EQUATION

Relative frequency The proportion of times the event occurs out of the number of times a random experiment is performed

147

Cumulative probability “Less than or equal to” probabilities associated with a random variable

150

Mean (or expected value) of a probability distribution

A measure of central tendency—the weighted sum of the possible values, weighted by their probabilities

Basic formulas

151 5.6

Variance of a probability distribution

A measure of variability: the weighted sum of the squared deviations of the possible values from the mean, weighted by the probabilities

Basic formulas

151 5.7, 5.9

Standard deviation of a probability distribution

A measure of variability: the square root of the variance Basic formulas

151 5.8

Density function Specifies the probability distribution of a continuous random variable

168

Normal distribution A continuous distribution with possible values ranging over the entire number line; its density function is a symmetric bell-shaped curve

169 5.1

Standardizing a normal random variable

Transforms any normal distribution with mean m and standard deviation s to the standard normal distribution with mean 0 and standard deviation 1

170 5.2

Normal calculations in Excel

Useful for finding probabilities and percentiles for non- standard and standard normal distributions

NORM.DIST, NORM.S.DIST, NORM.INV, NORM.S.INV

175

Empirical rules for normal distribution

About 68% of the data fall within one standard deviation of the mean, about 95% of the data fall within two stan- dard deviations of the mean, and almost all fall within three standard deviations of the mean

177

Binomial distribution The distribution of the number of successes in n inde- pendent, identical trials, where each trial has probability p of success

BINOM.DIST BINOM.INV

190

Mean and standard deviation of a binomial distribution

The mean and standard deviation of a binomial distri- bution with parameters n and p are np and !np 11 2 p2, respectively

193 5.3, 5.4

Sampling without replacement

Sampling where no member of the population can be sampled more than once

194

Hypergeometric distribution

Relevant when sampling without replacement, espe- cially when the fraction of the population sampled is large

HYPGEOM. DIST

Sampling with replacement

Sampling where any member of the population can be sampled more than once

194

Normal approximation to the binomial distribution

If np 7 5 and n11 2 p2 7 5, the binomial distribution can be approximated well by a normal distribution with mean np and standard deviation !np11 2 p2

194

Poisson distribution A discrete probability distribution that often describes the number of events occurring within a specified period of time or space; mean and variance both equal the parameter l

POISSON.DIST 207

Exponential distribution A continuous probability distribution useful for measur- ing times between events, such as customer arrivals to a service facility; mean and standard deviation both equal the reciprocal of the parameter

EXPON.DIST 210

Key Terms (continued)

09953_ch05_ptg01_183-241.indd 232 04/03/19 12:25 PM

5-7 Conclusion 2 3 3

Problems

Conceptual Questions C.1. Suppose that you want to find the probability that

event A or event B will occur. If these two events are not mutually exclusive, explain how you would proceed.

C.2. “If two events are mutually exclusive, they must not be independent events.” Is this statement true or false? Explain your choice.

C.3. Is the number of passengers who show up for a particu- lar commercial airline flight a discrete or a continuous random variable? Is the time between flight arrivals at a major airport a discrete or a continuous random vari- able? Explain your answers.

C.4. Suppose that officials in the federal government are trying to determine the likelihood of a major small- pox epidemic in the United States within the next 12 months. Is this an example of an objective probability or a subjective probability? How might the officials assess this probability?

C.5. Consider the statement, “When there are a finite num- ber of outcomes, then all probability is just a matter of counting. Specifically, if n of the outcomes are favor- able to some event E, and there are N outcomes total, then the probability of E is n/N .” Is this statement always true? Is it always false?

C.6. If there is uncertainty about some monetary outcome and you are concerned about return and risk, then all you need to see are the mean and standard deviation. The entire distribution provides no extra useful infor- mation. Do you agree or disagree? Provide an example to back up your argument.

C.7. Choose at least one uncertain quantity of interest to you. For example, you might choose the highest price of gas between now and the end of the year, the high- est point the Dow Jones Industrial Average will reach between now and the end of the year, the number of majors Tiger Woods will win in his career, and so on. Using all of the information and insight you have, assess the probability distribution of this uncertain quantity. Is there one “right answer?”

C.8. Historically, the most popular measure of variability has been the standard deviation, the square root of the weighted sum of squared deviations from the mean, weighted by their probabilities. Suppose analysts had always used an alternative measure of variability, the weighted sum of the absolute deviations from the mean, again weighted by their probabilities. Do you think this would have made a big difference in the the- ory and practice of probability and statistics?

C.9. Suppose a person flips a coin, but before you can see the result, the person puts her hand over the coin. At this point, does it make sense to talk about the probability

that the result is heads? Is this any different from the probability of heads before the coin was flipped?

C.10. Consider an event that will either occur or not. For example, the event might be that California will expe- rience a major earthquake in the next five years. You let p be the probability that the event will occur. Does it make any sense to have a probability distribution of p? Why or why not? If so, what might this distribution look like? How would you interpret it?

C.11. Suppose a couple is planning to have two children. Let B1 be the event that the first child is a boy, and let B2 be the event that the second child is a boy. You and your friend get into an argument about whether B1 and B2 are independent events. You think they are indepen- dent and your friend thinks they aren’t. Which of you is correct? How could you settle the argument?

C.12 For each of the following uncertain quantities, discuss whether it is reasonable to assume that the probabil- ity distribution of the quantity is normal. If the answer isn’t obvious, discuss how you could discover whether a normal distribution is reasonable. a. The change in the Dow Jones Industrial Average

between now and a year from now b. The length of time (in hours) a battery that is in con-

tinuous use lasts c. The time between two successive arrivals to a bank d. The time it takes a bank teller to service a random

customer e. The length (in yards) of a typical drive on a par 5 by

Phil Michelson f. The amount of snowfall (in inches) in a typical win-

ter in Minneapolis g. The average height (in inches) of all boys in a ran-

domly selected seventh-grade middle school class h. Your bonus from finishing a project, where your

bonus is $1000 per day under the deadline if the proj- ect is completed before the deadline, your bonus is $500 if the project is completed right on the dead- line, and your bonus is $0 if the project is completed after the deadline

i. Your gain on a call option on a stock, where you gain nothing if the price of the stock a month from now is less than or equal to $50 and you gain 1P 2 502 dollars if the price P a month from now is greater than $50

C.13 For each of the following uncertain quantities, discuss whether it is reasonable to assume that the probabil- ity distribution of the quantity is binomial. If you think it is, what are the parameters n and p? If you think it isn’t, explain your reasoning. a. The number of wins the Boston Red Sox baseball

team has next year in its 81 home games b. The number of free throws Kobe Bryant misses in

his next 250 attempts c. The number of free throws it takes Kobe Bryant to

achieve 100 successes

09953_ch05_ptg01_183-241.indd 233 04/03/19 12:25 PM

2 3 4 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

d. The number out of 1000 randomly selected custom- ers in a supermarket who have a bill of at least $150

e. The number of trading days in a typical year where Microsoft’s stock price increases

f. The number of spades you get in a 13-card hand from a well-shuffled 52-card deck

g. The number of adjacent 15-minute segments during a typical Friday where at least 10 customers enter a McDonald’s restaurant

h. The number of pages in a 500-page book with at least one misprint on the page

C.14 One disadvantage of a normal distribution is that there is always some probability that a quantity is negative, even when this makes no sense for the uncertain quan- tity. For example, the time a light bulb lasts cannot be negative. In any particular situation, how would you decide whether you could ignore this disadvantage for all practical purposes?

C.15 For real applications, the normal distribution has two potential drawbacks: (1) it can be negative, and (2) it isn’t symmetric. Choose some continuous random numeric outcomes of interest to you. Are either poten- tial drawbacks really drawbacks for your random out- comes? If so, which is the more serious drawback?

C.16 Many basketball players and fans believe strongly in the “hot hand.” That is, they believe that players tend to shoot in streaks, either makes or misses. If this is the case, why does the binomial distribution not apply, at least not exactly, to the number of makes in a given number of shots? Which assumption of the binomial model is violated, the independence of successive shots or the con- stant probability of success on each shot? Or can you tell?

Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 49. A business manager who needs to make many phone

calls has estimated that when she calls a client, the prob- ability that she will reach the client right away is 60%. If she does not reach the client on the first call, the proba- bility that she will reach the client with a subsequent call in the next hour is 20%. a. Find the probability that the manager reaches her

client in two or fewer calls. b. Find the probability that the manager reaches her

client on the second call but not on the first call. c. Find the probability that the manager is unsuccessful

on two consecutive calls. 50. Suppose that a marketing research firm sends

questionnaires to two different companies. Based on historical evidence, the marketing research firm believes that each company, independently of the other, will return the questionnaire with probability 0.40. a. What is the probability that both questionnaires are

returned?

b. What is the probability that neither of the question- naires is returned?

c. Now, suppose that this marketing research firm sends questionnaires to ten different companies. Assum- ing that each company, independently of the others, returns its completed questionnaire with probability 0.40, how do your answers to parts a and b change?

51. Based on past sales experience, an appliance store stocks five window air conditioner units for the coming week. No orders for additional air conditioners will be made until next week. The weekly consumer demand for this type of appliance has the probability distribution given in the file P05_51.xlsx. a. Let X be the number of window air conditioner units

left at the end of the week (if any), and let Y be the number of special stockout orders required (if any), assuming that a special stockout order is required each time there is a demand and no unit is available in stock. Find the probability distributions of X and Y.

b. Find the expected value of X and the expected value of Y.

c. Assume that this appliance store makes a $60 profit on each air conditioner sold from the weekly available stock, but the store loses $20 for each unit sold on a spe- cial stockout order basis. Let Z be the profit that the store earns in the coming week from the sale of window air conditioners. Find the probability distribution of Z.

d. Find the expected value of Z. 52. A roulette wheel contains the numbers 0, 00, and 1 to

36. If you bet $1 on a single number coming up, you earn $35 if the number comes up and lose $1 otherwise. Find the mean and standard deviation of your winnings on a single bet. Then find the mean and standard devia- tion of your net winnings if you make 100 bets. You can assume (realistically) that the results of the 100 spins are independent. Finally, provide an interval such that you are 95% sure your net winnings from 100 bets will be inside this interval.

53. You are involved in a risky business venture where three outcomes are possible: (1) you will lose not only your initial investment ($5000) but an additional $3000; (2) you will just make back your initial investment (for a net gain of $0); or (3) you will make back your initial investment plus an extra $10,000.

The probability of (1) is half as large as the probability of (2), and the probability of (3) is one-third as large as the probability of (2). a. Find the individual probabilities of (1), (2), and (3).

(They should sum to 1.) b. Find the expected value and standard deviation of

your net gain (or loss) from this venture. 54. Suppose the annual return on XYZ stock follows a nor-

mal distribution with mean 12% and standard deviation 30%. a. What is the probability that XYZ’s value will decrease

during a year?

09953_ch05_ptg01_183-241.indd 234 04/03/19 12:25 PM

5-7 Conclusion 2 3 5

b. What is the probability that the return on XYZ during a year will be at least 20%?

c. What is the probability that the return on XYZ during a year will be between 26% and 9%?

d. There is a 5% chance that the return on XYZ during a year will be greater than what value?

e. There is a 1% chance that the return on XYZ during a year will be less than what value?

f. There is a 95% chance that the return on XYZ during a year will be between which two values (equidistant from the mean)?

55. A family is considering a move from a midwestern city to a city in California. The distribution of housing costs where the family currently lives is normal, with mean $105,000 and standard deviation $18,200. The distri- bution of housing costs in the California city is normal with mean $235,000 and standard deviation $30,400. The family’s current house is valued at $110,000. a. What percentage of houses in the family’s current city

cost less than theirs? b. If the family buys a $200,000 house in the new city,

what percentage of houses there will cost less than theirs?

c. What price house will the family need to buy to be in the same percentile (of housing costs) in the new city as they are in the current city?

56. The number of traffic fatalities in a typical month in a given state has a normal distribution with mean 125 and standard deviation 31. a. If a person in the highway department claims that

there will be at least m fatalities in the next month with probability 0.95, what value of m makes this claim true?

b. If the claim is that there will be no more than n fatal- ities in the next month with probability 0.98, what value of n makes this claim true?

57. It can be shown that a sum of normally distributed ran- dom variables is also normally distributed. Do all func- tions of normal random variables lead to normal random variables? Consider the following. SuperDrugs is a chain of drugstores with three similar-size stores in a given city. The sales in a given week for any of these stores is normally distributed with mean $15,000 and standard deviation $3000. At the end of each week, the sales fig- ure for the store with the largest sales among the three stores is recorded. Is this maximum value normally dis- tributed? To answer this question, simulate a weekly sales figure at each of the three stores and calculate the maximum. Then replicate this maximum 500 times and create a histogram of the 500 maximum values. Does it appear to be normally shaped? Whatever this distribu- tion looks like, use your simulated values to estimate the mean and standard deviation of the maximum.

58. In the financial world, there are many types of com- plex instruments called derivatives that derive their value from the value of an underlying asset. Consider

the following simple derivative. A stock’s current price is $80 per share. You purchase a derivative whose value to you becomes known a month from now. Specifi- cally, let P be the price of the stock in a month. If P is between $75 and $85, the derivative is worth noth- ing to you. If P is less than $75, the derivative results in a loss of 100*175 2 P2 dollars to you. (The factor of 100 is because many derivatives involve 100 shares.) If P is greater than $85, the derivative results in a gain of 100*1P 2 852 dollars to you. Assume that the distri- bution of the change in the stock price from now to a month from now is normally distributed with mean $1 and standard deviation $8. Let P (big loss) be the proba- bility that you lose at least $1000 (that is, the price falls below $65), and let P (big gain) be the probability that you gain at least $1000 (that is, the price rises above $95). Find these two probabilities. How do they com- pare to one another?

Level B 59. Equation (5.7) for variance indicates exactly what vari-

ance is: the weighted average of squared deviations from the mean, weighted by the probabilities. However, the computing formula for variance, Equation (5.9), is more convenient for spreadsheet calculations. Show algebra- ically that the two formulas are equivalent.

60. The basic game of craps works as follows. You throw two dice. If the sum of the two faces showing up is 7 or 11, you win and the game is over. If the sum is 2, 3, or 12, you lose and the game is over. If the sum is anything else (4, 5, 6, 8, 9, or 10), that value becomes your “point.” You then keep throwing the dice until the sum matches your point or equals 7. If your point occurs first, you win and the game is over. If 7 occurs first, you lose and the game is over. What is the probability that you win the game?

61. Consider an individual selected at random from a sample of 750 married women (see the data in the file P05_61. xlsx) in answering each of the following questions. a. What is the probability that this woman does not work

outside the home, given that she has at least one child? b. What is the probability that this woman has no chil-

dren, given that she works part time? c. What is the probability that this woman has at least

two children, given that she does not work full time? 62. Suppose that 8% of all managers in a given company

are African American, 13% are women, and 17% have earned an MBA degree from a top-10 graduate business school. Let A, B, and C be, respectively, the events that a randomly selected individual from this population is African American, is a woman, and has earned an MBA from a top-10 graduate business school. a. Do you believe that A, B, and C are independent

events? Explain why or why not.

09953_ch05_ptg01_183-241.indd 235 04/03/19 12:25 PM

2 3 6 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

b. Assuming that A, B, and C are independent events, find the probability that a randomly selected manager from this company is a white male and has earned an MBA degree from a top-10 graduate business school.

c. If A, B, and C are not independent events, can you calculate the probability requested in part b from the information given? What further information would you need?

63. Two gamblers play a version of roulette with a wheel as shown in the file P05_63.xlsx. Each gambler places four bets, but their strategies are different, as explained below. For each gambler, use the rules of probability to find the distribution of their net winnings after four bets. Then find the mean and standard deviation of their net winnings. The file gets you started. a. Player 1 always bets on red. On each bet, he either

wins or loses what he bets. His first bet is for $10. From then on, he bets $10 following a win, and he doubles his bet after a loss. (This is called a martingale strategy and is used frequently at casinos.) For example, if he spins red, red, not red, and not red, his bets are for $10, $10, $10, and $20, and he has a net loss of $10. Or if he spins not red, not red, not red, and red, then his bets are for $10, $20, $40, and $80, and he has a net gain of $10.

b. Player 2 always bets on black and green. On each bet, he places $10 on black and $2 on green. If red occurs, he loses all $12. If black occurs, he wins a net $8 ($10 gain on black, $2 loss on green). If green occurs, he wins a net $50 ($10 loss on black, $60 gain on green).

64. Suppose the New York Yankees and Philadelphia Phil- lies (two Major League Baseball teams) are playing a best-of-three series. The first team to win two games is the winner of the series, and the series ends as soon as one team has won two games. The first game is played in New York, the second game is in Phil- adelphia, and if necessary the third game is in New York. The probability that the Yankees win a game in their home park is 0.55. The probability that the Phil- lies win a game in their home park is 0.53. You can assume that the outcomes of the games are probabilis- tically independent. a. Find the probability that the Yankees win the series. b. Suppose you are a Yankees fan, so you place a bet

on each game played where you win $100 if the Yankees win the game and you lose $105 if the Yankees lose the game. Find the distribution of your net winnings. Then find the mean and standard deviation of this distribution. Is this betting strategy favorable to you?

c. Repeat part a, but assume that the games are played in Philadelphia, then New York, then Philadelphia. How much does this “home field advantage” help the Phillies?

d. Repeat part a, but now assume that the series is a best- of-five series, where the first team that wins three games wins the series. Assume that games alternate between New York and Philadelphia, with the first game in New York.

65. Have you ever watched the odds at a horse race? You might hear that the odds against a given horse win- ning are 9 to 1, meaning that the horse has probability 1>11 1 92 5 1>10 of winning. However, these odds, after being converted to probabilities, typically sum to something greater than one. Why is this? Suppose you place a bet of $10 on this horse. It seems that it is a fair bet if you lose your $10 if the horse loses, but you win $90 if the horse wins. However, argue why this isn’t really fair to you, that is, argue why your expected win- nings are negative.

66. When you sum 30 or more independent random vari- ables, the sum of the random variables will usually be approximately normally distributed, even if each individual random variable is not normally distributed. Use this fact to estimate the probability that a casino will be behind after 90,000 roulette bets, given that it wins $1 or loses $35 on each bet with probabilities 37/38 and 1/38.

67. Many companies use sampling to determine whether a batch should be accepted. An 1n, c2 sampling plan con- sists of inspecting n randomly chosen items from a batch and accepting the batch if c or fewer sampled items are defective. Suppose a company uses a 1100, 52 sampling plan to determine whether a batch of 10,000 computer chips is acceptable. a. The “producer’s risk” of a sampling plan is the prob-

ability that an acceptable batch will be rejected by the sampling plan. Suppose the customer considers a batch with 3% defectives acceptable. What is the pro- ducer’s risk for this sampling plan?

b. The “consumer’s risk” of a sampling plan is the prob- ability that an unacceptable batch will be accepted by the sampling plan. Our customer says that a batch with 9% defectives is unacceptable. What is the con- sumer’s risk for this sampling plan?

68. Suppose that if a presidential election were held today, 53% of all voters would vote for candidate Smith over candidate Jones. (You can substitute the names of the most recent presidential candidates.) This problem shows that even if there are 100 million voters, a sample of several thousand is enough to determine the outcome, even in a fairly close election. a. If 1500 voters are sampled randomly, what is the

probability that the sample will indicate (correctly) that Smith is preferred to Jones?

b. If 6000 voters are sampled randomly, what is the probability that the sample will indicate (correctly) that Smith is preferred to Jones?

09953_ch05_ptg01_183-241.indd 236 04/03/19 12:25 PM

5-7 Conclusion 2 3 7

69. A company assembles a large part by joining two smaller parts together. Assume that the smaller parts are normally distributed with a mean length of 1 inch and a standard deviation of 0.01 inch. a. What fraction of the larger parts are longer than 2.05

inches? b. What fraction of the larger parts are between 1.96

inches and 2.02 inches long? 70. (Suggested by Sam Kaufmann, Indiana University MBA,

who has run Harrah’s Lake Tahoe Casino.) A high roller has come to the casino to play 300 games of craps. For each game of craps played there is a 0.493 probability that the high roller will win $1 and a 0.507 probabil- ity that the high roller will lose $1. After 300 games of craps, what is the probability that the casino will be behind more than $10?

71. (Suggested by Sam Kaufmann, Indiana University MBA, who has run Harrah’s Lake Tahoe Casino.) A high roller comes to the casino intending to play 500 hands of blackjack for $1 a hand. On each hand, the high roller will win $1 with probability 0.48 and lose $1 with prob- ability 0.52. After the 500 hands, what is the probability that the casino has lost more than $40?

72. The weekly demand for TVs at Lowland Appliance is normally distributed with mean 400 and standard devia- tion 100. Each time an order for TVs is placed, it arrives exactly four weeks later. That is, TV orders have a four- week lead time. Lowland doesn’t want to run out of TVs during any more than 1% of all lead times. How low should Lowland let its TV inventory drop before it places an order for more TVs? (Hint: How many stan- dard deviations above the mean lead-time demand must the reorder point be for there to be a 1% chance of a stockout during the lead time?)

73. An elevator rail is assumed to meet specifications if its diameter is between 0.98 and 1.01 inches. Each year a company produces 100, 000 elevator rails. For a cost of $10/s2 per year the company can rent a machine that produces elevator rails whose diame- ters have a standard deviation of s. (The idea is that the company must pay more for a smaller variance.) Each such machine will produce rails having a mean diameter of one inch. Any rail that does not meet spec- ifications must be reworked at a cost of $12. Assume that the diameter of an elevator rail follows a normal distribution. a. What standard deviation (within 0.001 inch) mini-

mizes the annual cost of producing elevator rails? You do not need to try standard deviations in excess of 0.02 inch.

b. For your answer in part a, one elevator rail in 1000 will be at least how many inches in diameter?

74. What caused the crash of TWA Flight 800 in 1996? Physics professors Hailey and Helfand of Columbia University

believe there is a reasonable possibility that a meteor hit Flight 800. They reason as follows. On a given day, 3000 meteors of a size large enough to destroy an airplane hit the earth’s atmosphere. Approximately 50,000 flights per day, averaging two hours in length, have been flown from 1950 to 1996. This means that at any given point in time, planes in flight cover approximately two-billionths of the world’s atmosphere. Determine the probability that at least one plane in the last 47 years has been downed by a meteor. (Hint: Use the Poisson approximation to the binomial. This approximation says that if n is large and p is small, a binomial distribution with parameters n and p is approximately Poisson distributed with l 5 np.)

75. In the decade 1982 through 1991, 10 employees work- ing at the Amoco Company chemical research center were stricken with brain tumors. The average employ- ment at the center was 2000 employees. Nationwide, the average incidence of brain tumors in a single year is 20 per 100,000 people. If the incidence of brain tumors at the Amoco chemical research center were the same as the nationwide incidence, what is the probability that at least 10 brain tumors would have been observed among Amoco workers during the decade 1982 through 1991? What do you conclude from your analysis? (Source: AP wire service report, March 12, 1994)

76. Claims arrive at random times to an insurance company. The daily amount of claims is normally distributed with mean $1570 and standard deviation $450. Total claims on different days each have this distribution, and they are probabilistically independent of one another. a. Find the probability that the amount of total claims

over a period of 100 days is at least $150,000. b. If the company receives premiums totaling $165,000,

find the probability that the company will net at least $10,000 for the 100-day period.

77. Your company is running an audit on Sleaze Company. Because Sleaze has a bad habit of overcharging its cus- tomers, the focus of your audit is on checking whether the billing amounts on its invoices are correct. Assume that each invoice is for too high an amount with prob- ability 0.06 and for too low an amount with probabil- ity 0.01 (so that the probability of a correct billing is 0.93). Also, assume that the outcome for any invoice is probabilistically independent of the outcomes for other invoices. a. If you randomly sample 200 of Sleaze’s invoices,

what is the probability that you will find at least 15 invoices that overcharge the customer? What is the probability you won’t find any that undercharge the customer?

b. Find an integer k such that the probability is at least 0.99 that you will find at least k invoices that over- charge the customer. (Hint: Use trial and error with the BINOM.DIST function to find k.)

09953_ch05_ptg01_183-241.indd 237 04/03/19 12:26 PM

2 3 8 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

78. Continuing the previous problem, suppose that when Sleaze overcharges a customer, the distribution of the amount overcharged (expressed as a percentage of the correct billing amount) is normally distributed with mean 15% and standard deviation 4%. a. What percentage of overbilled customers are charged

at least 10% more than they should pay? b. What percentage of all customers are charged at least

10% more than they should pay? c. If your auditing company samples 200 randomly cho-

sen invoices, what is the probability that it will find at least five where the customer was overcharged by at least 10%?

79. As any credit-granting agency knows, there are always some customers who default on credit charges. Typically, customers are grouped into relatively homogeneous categories, so that customers within any category have approximately the same chance of defaulting on their credit charges. Here we will look at one particular group of customers. We assume each of these customers has (1) probability 0.07 of defaulting on his or her current credit charges, and (2) total credit charges that are nor- mally distributed with mean $350 and standard devia- tion $100. We also assume that if a customer defaults, 20% of his or her charges can be recovered. The other 80% are written off as bad debt. a. What is the probability that a typical customer in this

group will default and produce a write-off of more than $250 in bad debt?

b. If there are 500 customers in this group, what are the mean and standard deviation of the number of cus- tomers who will meet the description in part a?

c. Again assuming there are 500 customers in this group, what is the probability that at least 25 of them will meet the description in part a?

d. Suppose now that nothing is recovered from a default—the whole amount is written off as bad debt. Show how to simulate the total amount of bad debt from 500 customers in just two cells, one with a binomial calculation, the other with a normal calculation.

80. The Excel functions discussed in this chapter are use- ful for solving a lot of probability problems, but there are other problems that, even though they are similar to normal or binomial problems, cannot be solved with these functions. In cases like this, simulation can often be used. Here are a couple of such problems for you to simulate. For each example, simulate 500 replications of the experiment. a. You observe a sequence of parts from a manufactur-

ing line. These parts use a component that is supplied

by one of two suppliers. Each part made with a com- ponent from supplier 1 works properly with prob- ability 0.95, and each part made with a component from supplier 2 works properly with probability 0.98. Assuming that 100 of these parts are made, 60 from supplier 1 and 40 from supplier 2, you want the probability that at least 97 of them work properly.

b. Here we look at a more generic example such as coin flipping. There is a sequence of trials where each trial is a success with probability p and a failure with prob- ability 1 2 p. A run is a sequence of consecutive suc- cesses or failures. For most of us, intuition says that there should not be long runs. Test this by finding the probability that there is at least one run of length at least six in a sequence of 15 trials. (The run could be of 0s or 1s.) You can use any value of p you like—or try different values of p.

81. In the game of soccer, players are sometimes awarded a penalty kick. The player who kicks places the ball 12 yards from the 24-foot-wide goal and attempts to kick it past the goalie into the net. (The goalie is the only defender.) The question is where the player should aim. Make the following assumptions. (1) The player’s kick is off target from where he aims, left or right, by a normally distributed amount with mean 0 and some standard devi- ation. (2) The goalie typically guesses left or right and dives in that direction at the moment the player kicks. If the goalie guesses wrong, he won’t block the kick, but if he guesses correctly, he will be able to block a kick that would have gone into the net as long as the kick is within a distance d from the middle of the goal. The goalie is equally likely to guess left or right. (3) The player never misses high, but he can miss to the right of the goal (if he aims to the right) or to the left (if he aims to the left). For reasonable values of the standard deviation and d, find the probability that the player makes a goal if he aims at a point t feet inside the goal. (By symmetry, you can assume he aims to the right, although the goalie doesn’t know this.) What value of t seems to maximize the prob- ability of making a goal?

82. In the 2012 Major League Baseball season, the Bal- timore Orioles were ahead after the 7th inning in 74 games, and they won all 74 games. Use an appro- priate model to explore how unusual such a streak is. Would you place it in the same category as the famous 56-game hitting streak (at least one hit per game) by Joe DiMaggio in 1941? Discuss the differences, including those that are caused by pressure.

09953_ch05_ptg01_183-241.indd 238 04/03/19 12:26 PM

5-7 Conclusion 2 3 9

CASE 5.2 EuroWatch Company EuroWatch Company assembles expensive wristwatches and then sells them to retailers throughout Europe. The watches are assembled at a plant with two assembly lines. These lines are intended to be identical, but line 1 uses somewhat older equipment than line 2 and is typically less reliable. Histori- cal data have shown that each watch coming off line 1, inde- pendently of the others, is free of defects with probability 0.98. The similar probability for line 2 is 0.99. Each line produces 500 watches per hour. The production manager has asked you to answer the following questions.

1. She wants to know how many defect-free watches each line is likely to produce in a given hour. Specifically, find the smallest integer k (for each line separately) such that you can be 99% sure that the line will not produce more than k defective watches in a given hour.

2. EuroWatch currently has an order for 500 watches from an important customer. The company plans to fill this

order by packing slightly more than 500 watches, all from line 2, and sending this package off to the cus- tomer. Obviously, EuroWatch wants to send as few watches as possible, but it wants to be 99% sure that when the customer opens the package, there are at least 500 defect-free watches. How many watches should be packed?

3. EuroWatch has another order for 1000 watches. Now it plans to fill this order by packing slightly more than one hour’s production from each line. This package will contain the same number of watches from each line. As in the previous question, EuroWatch wants to send as few watches as possible, but it again wants to be 99% sure that when the customer opens the package, there are at least 1000 defect-free watches. The question of how many watches to pack is unfortunately quite diffi- cult because the total number of defect-free watches is not binomially distributed. (Why not?) Therefore, the

CASE 5.1 Simpson’s Paradox The results we obtain with conditional probabilities can be quite counterintuitive, even paradoxical. This case is similar to one described in an article by Blyth (1972), and is usually referred to as Simpson’s paradox. [Two other examples of Simpson’s paradox are described in articles by Westbrooke (1998) and Appleton et al. (1996).] Essentially, Simpson’s paradox says that even if one treatment has a better effect than another on each of two separate subpopulations, it can have a worse effect on the population as a whole.

Suppose that the population is the set of managers in a large company. We categorize the managers as those with an MBA degree (the Bs) and those without an MBA degree (the Bs). These categories are the two “treatment” groups. We also categorize the managers as those who were hired directly out of school by this company (the Cs) and those who worked with another company first (the Cs). These two categories form the two “subpopulations.” Finally, we use as a measure of effectiveness those managers who have been promoted within the past year (the As).

Assume the following conditional probabilities are given:

P1AuB and C2 5 0.10, P1AuB and C2 5 0.05 (5.16)

P1AuB and C2 5 0.35, P1AuB and C2 5 0.20 (5.17)

P1CuB2 5 0.90, P1CuB2 5 0.30 (5.18) Each of these can be interpreted as a proportion. For exam- ple, the probability P1AuB and C2 implies that 10% of all

managers who have an MBA degree and were hired by the company directly out of school were promoted last year. Similar explanations hold for the other probabilities.

Joan Seymour, the head of personnel at this company, is trying to understand these figures. From the proba- bilities in Equation (5.16), she sees that among the sub- population of workers hired directly out of school, those with an MBA degree are twice as likely to be promoted as those without an MBA degree. Similarly, from the prob- abilities in Equation (5.17), she sees that among the sub- population of workers hired after working with another company, those with an MBA degree are almost twice as likely to be promoted as those without an MBA degree. The information provided by the probabilities in Equation (5.18) is somewhat different. From these, she sees that employees with MBA degrees are three times as likely as those without MBA degrees to have been hired directly out of school.

Joan can hardly believe it when a whiz-kid analyst uses these probabilities to show—correctly—that

P1AuB2 5 0.125, P1AuB2 5 0.155 (5.19) In words, those employees without MBA degrees are more likely to be promoted than those with MBA degrees. This appears to go directly against the evidence in Equations (5.16) and (5.17), both of which imply that MBAs have an advantage in being promoted. Can you derive the proba- bilities in Equation (5.19)? Can you shed any light on this “paradox”?

09953_ch05_ptg01_183-241.indd 239 04/03/19 12:26 PM

2 4 0 C H A P T E R 5 P r o b a b i l i t y a n d P r o b a b i l i t y D i s t r i b u t i o n s

manager asks you to solve the problem with simulation (and some trial and error). (Hint: It turns out that it is much faster to simulate small numbers than large num- bers, so simulate the number of watches with defects, not the number without defects.)

4. Finally, EuroWatch has a third order for 100 watches. The customer has agreed to pay $50,000 for the order— that is, $500 per watch. If EuroWatch sends more than 100 watches to the customer, its revenue doesn’t increase; it can never exceed $50,000. Its unit cost of producing a watch is $450, regardless of which line it is assembled on. The order will be filled entirely from a single line, and EuroWatch plans to send slightly more than 100 watches to the customer.

If the customer opens the shipment and finds that there are fewer than 100 defect-free watches (which we assume the customer has the ability to do), then he will pay only for the defect-free watches—EuroWatch’s revenue will decrease by $500 per watch short of the 100 required—and on top of this, EuroWatch will be

required to make up the difference at an expedited cost of $1000 per watch. The customer won’t pay a dime for these expedited watches. (If expediting is required, EuroWatch will make sure that the expedited watches are defect-free. It doesn’t want to lose this customer entirely.)

You have been asked to develop a spreadsheet model to find EuroWatch’s expected profit for any number of watches it sends to the customer. You should develop it so that it responds correctly, regardless of which assem- bly line is used to fill the order and what the shipment quantity is. (Hints: Use the BINOM.DIST function, with last argument 0, to fill up a column of probabilities for each possible number of defective watches. Next to each of these, calculate EuroWatch’s profit. Then use a SUM- PRODUCT to obtain the expected profit. Finally, you can assume that EuroWatch will never send more than 110 watches. It turns out that this large a shipment is not even close to optimal.)

CASE 5.3 Cashing in on the Lottery Many states supplement their tax revenues with state-spon- sored lotteries. Most of them do so with a game called lotto. Although there are various versions of this game, they are all basically as follows. People purchase tickets that contain r distinct numbers from 1 to m, where r is generally 5 or 6 and m is generally around 50. For example, in Virginia, the state discussed in this case, r 5 6 and m 5 44. Each ticket costs $1, about 39 cents of which is allocated to the total jackpot.6 There is eventually a drawing of r 5 6 distinct numbers from the m 5 44 possible numbers. Any ticket that matches these 6 numbers wins the jackpot.

There are two interesting aspects of this game. First, the current jackpot includes not only the revenue from this round of ticket purchases but also any jackpots carried over from previous drawings because of no winning tickets. Therefore, the jackpot can build from one drawing to the next, and in celebrated cases it has become huge. Second, if there is more than one winning ticket—a distinct possibility—the winners share the jackpot equally. (This is called parimutuel betting.) So, for example, if the current jackpot is $9 million and there are three winning tickets, then each winner receives $3 million.

It can be shown that for Virginia’s choice of r and m, there are approximately 7 million possible tickets (7,059,052 to be exact). Therefore, any ticket has about one chance out

of 7 million of being a winner. That is, the probability of winning with a single ticket is p 5 1/7,059,052—not very good odds. If n people purchase tickets, then the number of winners is binomially distributed with parameters n and p. Because n is typically very large and p is small, the num- ber of winners has approximately a Poisson distribution with rate l 5 np. (This makes ensuing calculations somewhat easier.) For example, if 1 million tickets are purchased, then the number of winning tickets is approximately Poisson dis- tributed with l 5 1>7.

In 1992, an Australian syndicate purchased a huge num- ber of tickets in the Virginia lottery in an attempt to assure itself of purchasing a winner. It worked! Although the syn- dicate wasn’t able to purchase all 7 million possible tickets (it was about 1.5 million shy of this), it did purchase a win- ning ticket, and there were no other winners. Therefore, the syndicate won a 20-year income stream worth approximately $27 million, with a net present value of approximately $14 million. This made the syndicate a big profit over the cost of the tickets it purchased. Two questions come to mind: (1) Is this hogging of tickets unfair to the rest of the public? (2) Is it a wise strategy on the part of the syndicate (or did it just get lucky)?

To answer the first question, consider how the lottery changes for the general public with the addition of the syn- dicate. To be specific, suppose the syndicate can invest $7 million and obtain all of the possible tickets, making itself a sure winner. Also, suppose n people from the general public purchase tickets, each of which has 1 chance out of 7 mil- lion of being a winner. Finally, let R be the jackpot carried

6 Of the remaining 61 cents, the state takes about 50 cents. The other 11 cents is used to pay off lesser prize winners whose tickets match some, but not all, of the winning 6 numbers. To keep this case relatively simple, however, we ignore these lesser prizes and concentrate only on the jackpot.

09953_ch05_ptg01_183-241.indd 240 04/03/19 12:26 PM

5-7 Conclusion 2 4 1

over from any previous lotteries. Then the total jackpot on this round will be 3R 1 0.39 17,000,000 1 n24 because 39 cents from every ticket goes toward the jackpot. The number of winning tickets for the public will be Poisson distributed with l 5 n>7,000,000. However, any member of the public who wins will necessarily have to share the jackpot with the syndicate, which is a sure winner. Use this information to calculate the expected amount the public will win. Then do the same calculation when the syndicate does not play. (In this case the jackpot will be smaller, but the public won’t have to share any winnings with the syndicate.) For values of n and R that you can select, is the public better off with or without the syndicate? Would you, as a general member of the public, support a move to outlaw syndicates from hog- ging the tickets?

The second question is whether the syndicate is wise to buy so many tickets. Again assume that the syndicate can spend $7 million and purchase each possible ticket. (Would this be possible in reality?) Also, assume that n members of the general public purchase tickets, and that the carryover

from the previous jackpot is R. The syndicate is thus assured of having a winning ticket, but is it assured of covering its costs? Calculate the expected net benefit (in terms of net present value) to the syndicate, using any reasonable values of n and R, to see whether the syndicate can expect to come out ahead.

Actually, the analysis suggested in the previous para- graph is not complete. There are at least two complications to consider. The first is the effect of taxes. Fortunately for the Australian syndicate, it did not have to pay federal or state taxes on its winnings, but a U.S. syndicate wouldn’t be so lucky. Second, the jackpot from a $20 million jackpot, say, is actually paid in 20 annual $1 million payments. The Lot- tery Commission pays the winner $1 million immediately and then purchases 19 “strips” (bonds with the interest not included) maturing at 1-year intervals with face value of $1 million each. Unfortunately, the lottery prize does not offer the liquidity of the Treasury issues that back up the payments. This lack of liquidity could make the lottery less attractive to the syndicate.

09953_ch05_ptg01_183-241.indd 241 04/03/19 12:26 PM

COST- EFFECTIVE HEPATITIS B INTERVENTIONS Hepatitis B is a viral disease that can lead to death and liver cancer if not treated. It is especially prevalent in Asian populations. The disease chronically infects approximately 8% to 10% of people in China and a similar percentage of Americans of Asian descent. It often infects newborns and children, in which case it is likely to become a life- long infection. Chronic infection is often asymptomatic for decades, but if left untreated, about 25% of the chronically infected will die of liver diseases such as cirrhosis or liver cancer.

A hepatitis B vaccine became available in the 1980s, but it is costly (thousands of dollars per year) and it does not cure the disease. Vaccination of children in the United States is widespread, and only about 0.5% of the general popu- lation is infected. This percentage, however, jumps to about 10% for U.S. adult Asian and Pacific Islanders, where the rate of liver cancer is more than three times that of the general U.S. population. The situation is even worse in China, where it is estimated that approxi- mately 300,000 die each year from liver disease caused by hepatitis B. Although rates of newborn vaccination in China have increased in recent years, about 20% of 1- to 4-year olds and 40% of 5- to 19-year olds still remain unprotected. In a pilot program in the Qinghai province, the feasibility of a vaccination “catch-up” program was demonstrated, but China’s public health officials worried about the cost effectiveness of a country-wide catch-up program.

The article by Hutton et al. (2011) reports the results of the work his team carried out over several years with the Asian Liver Center at Stanford University. They used decision analysis and other quantitative methods to analyze the cost effectiveness of several inter- ventions to combat hepatitis B in the United States and China. They addressed two policy questions in the study: (1) What combination of screening, treatment, and vaccination is most cost effective for U.S. adult Asian and Pacific Islanders; and (2) Is it cost effective to provide hepatitis B catch-up vaccination for children and adolescents in China?

For the first question, the team first considered the approach usually favored by the medical community, clinical trials, but they decided it was infeasible because of expense and the time (probably decades) required. Instead, they used decision analysis with a deci- sion tree very much like those in this chapter. The initial decision is whether to screen people for the disease. If this initial decision is no, the next decision is whether to vacci- nate. If this decision is no, they wait to see whether infection occurs. On the other side, if the initial decision to screen is yes, there are three possible outcomes: infected, immune, or susceptible. If infected, the next decision is whether to treat. If immune, no action is necessary. If susceptible, the sequence is the same as after the “no screen” decision. At end nodes with a possibility (or certainty) of being infected, the team used a probabilistic “Markov” model to determine future health states.

The various decision strategies were compared in terms of incremental cost to gain an incremental unit of health, with units of health expressed in quality-adjusted life years (QALYs). Interventions are considered cost effective if they cost less per QALY gained than three times a country’s per capita GDP; they are considered very cost effective if they cost less than a country’s per capita GDP.

CHAPTER 6 Decision Making Under Uncertainty

w av

eb re

ak m

ed ia

/S hu

tt er

st oc

k. co

09953_ch06_ptg01_242-292.indd 242 04/03/19 6:13 PM

6-1 Introduction 2 4 3

The study found that it is cost effective to screen Asian and Pacific Islanders so that they can receive treatment, and it is also cost effective to vaccinate those in close contact with infected individuals so that they can be protected from infection. Specifically, they estimated that this policy costs from $36,000 to $40,000 per QALY gained, whereas an intervention that costs $50,000 per QALY gained is considered cost effective in the United States. However, they found that it is not cost effective to provide universal vaccination for all U.S. adult Asian and Pacific Islanders, primarily because the risk of being exposed to hepatitis B for U.S. adults is low.

For the second question, the team used a similar decision analysis to determine that providing catch-up vaccination for children up to age 19 not only improves health out- comes but saves costs. Using sensitivity analysis, they found that catch-up vaccination might not be cost saving if the probability of a child becoming infected is one-fifth as high as the base-case estimate of 100 out of 100,000 per year. This is due to the high level of newborn vaccination coverage already achieved in some urban areas of China. They also found if treatment becomes cheaper, the cost advantages of vaccination decrease. However, treatment costs would have to be halved and infection risk would have to be five times lower than in their base case before the cost of providing catch-up vaccination would exceed $2500 per QALY gained (roughly equal to per capita GDP in China).

In any case, their analysis influenced China’s 2009 decision to expand free catch-up vaccination to all children in China under the age of 15. This decision could result in about 170 million children being vaccinated, and it could prevent hundreds of thousands of chronic infections and close to 70,000 deaths from hepatitis B.

6-1 Introduction This chapter provides a formal framework for analyzing decision problems that involve uncertainty. Our discussion includes the following:

• criteria for choosing among alternative decisions • how probabilities are used in the decision-making process • how early decisions affect decisions made at a later stage • how a decision maker can quantify the value of information • how attitudes toward risk can affect the analysis

Throughout, we employ a powerful graphical tool—a decision tree—to guide the analysis. A decision tree enables a decision maker to view all important aspects of the problem at once: the decision alternatives, the uncertain outcomes and their probabilities, the eco- nomic consequences, and the chronological order of events. Although decision trees have been used for years, often created with paper and pencil, we show how they can be imple- mented in Excel with the PrecisionTree add-in from Palisade.

Many examples of decision making under uncertainty exist in the business world, including the following:

• Companies routinely place bids for contracts to complete a certain project within a fixed time frame. Often these are sealed bids, where each company presents a bid for complet- ing the project in a sealed envelope. Then the envelopes are opened, and the low bidder is awarded the bid amount to complete the project. Any particular company in the bidding competition must deal with the uncertainty of the other companies’ bids, as well as possible uncertainty regarding their cost to complete the project if they win the bid. The trade-off is between bidding low to win the bid and bidding high to make a larger profit.

• Whenever a company contemplates introducing a new product into the market, there are a number of uncertainties that affect the decision, probably the most important being the customers’ reaction to this product. If the product generates high customer demand, the company will make a large profit. But if demand is low—and the vast majority of new products do poorly—the company could fail to recoup its development costs. Because the level of customer demand is critical, the company might try to gauge this level by

09953_ch06_ptg01_242-292.indd 243 04/03/19 6:13 PM

2 4 4 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

test marketing the product in one region of the country. If this test market is a success, the company can then be more optimistic that a full-scale national marketing of the product will also be successful. But if the test market is a failure, the company can cut its losses by abandoning the product.

• Whenever manufacturing companies make capacity expansion decisions, they face uncertain consequences. First, they must decide whether to build new plants. If they don’t expand and demand for their products is higher than expected, they will lose rev- enue because of insufficient capacity. If they do expand and demand for their prod- ucts is lower than expected, they will be stuck with expensive underutilized capacity. Companies also need to decide where to build new plants. This decision involves a whole new set of uncertainties, including exchange rates, labor availability, social stabil- ity, competition from local businesses, and others.

• Banks must continually make decisions on whether to grant loans to businesses or indi- viduals. Many banks made many very poor decisions, especially on mortgage loans, during the years leading up to the financial crisis in 2008. They fooled themselves into thinking that housing prices would only increase, never decrease. When the bottom fell out of the housing market, banks were stuck with loans that could never be repaid.

• Utility companies must make many decisions that have significant environmental and economic consequences. For these companies it is not necessarily enough to conform to federal or state environmental regulations. Recent court decisions have found com- panies liable—for huge settlements—when accidents occurred, even though the compa- nies followed all existing regulations. Therefore, when utility companies decide whether to replace equipment or mitigate the effects of environmental pollution, they must take into account the possible environmental consequences (such as injuries to people) as well as economic consequences (such as lawsuits). An aspect of these situations that makes decision analysis particularly difficult is that the potential “disasters” are often extremely unlikely; hence, their probabilities are difficult to assess accurately.

• Sports teams continually make decisions under uncertainty. Sometimes these decisions involve long-run consequences, such as whether to trade for a promising but as yet untested pitcher in baseball. Other times these decisions involve short-run consequences, such as whether to go for a fourth down or kick a field goal late in a close football game. You might be surprised at the level of quantitative sophistication in today’s professional sports. Management and coaches typically do not make important decisions by gut feel- ing. They employ many of the tools in this chapter and in other chapters of this book.

Although the focus of this chapter is on business decisions, the approach discussed in this chapter can also be used in important personal decisions you have to make. As an example, if you are just finishing an undergraduate degree, should you go immediately into a graduate program, or should you work for several years and then decide whether to pursue a graduate degree? As another example, if you currently have a decent job but you have the option to take another possibly more promising job that would require you and your family to move to another part of the country, should you stay or move?

You might not have to make too many life-changing decisions like these, but you will undoubtedly have to make a few. How will you make them? You will probably not use all the formal methods discussed in this chapter, but the discussion provided here should at least motivate you to think in a structured way before making your final decisions.

6-2 Elements of Decision Analysis Although decision making under uncertainty occurs in a wide variety of contexts, the problems we discuss in this chapter are alike in the following ways:

1. A problem has been identified that requires a solution. 2. A number of possible decisions have been identified. 3. Each decision leads to a number of possible outcomes.

09953_ch06_ptg01_242-292.indd 244 04/03/19 6:13 PM

6-2 elements of Decision analysis 2 4 5

4. There is uncertainty about which outcome will occur, and probabilities of the possible outcomes are assessed.

5. For each decision and each possible outcome, a payoff is received or a cost is incurred. 6. A “best” decision must be chosen using an appropriate decision criterion.

We now discuss these elements in some generality.1

Identifying the Problem When something triggers the need to solve a problem, you should think carefully about the problem that needs to be solved before diving in. Perhaps you are just finishing your undergrad- uate degree (the trigger), and you want to choose the Business School where you should get your MBA degree. You could define the problem as which MBA program you should attend, but maybe you should define it more generally as what you should do next now that you have your undergraduate degree. You don’t necessarily have to enter an MBA program right away. You could get a job and then get an MBA degree later, or you could enter a graduate program in some area other than Business. Maybe you could even open your own business and forget about graduate school. The point is that by changing the problem from deciding which MBA program to attend to deciding what to do next, you change the decision problem in a fundamental way.

Possible Decisions The possible decisions depend on the previous step: how the problem is specified. But after you identify the problem, all possible decisions for this problem should be listed. Keep in mind that if a potential decision isn’t in this list, it won’t have a chance of being chosen as the best decision later, so this list should be as comprehensive as possible. Some problems are of a multistage nature, as discussed in Section 6.6. In such problems, a first-stage decision is made, then an uncertain outcome is observed, then a second-stage decision is made, then a second uncertain outcome is observed, and so on. (Often there are only two stages, but there could be more.) In this case, a “decision” is really a “strategy” or “contingency plan” that prescribes what to do at each stage, depending on prior deci- sions and observed outcomes. These ideas are clarified in Section 6.6.

Possible Outcomes One of the main reasons why decision making under uncertainty is difficult is that decisions have to be made before uncertain outcomes are revealed. For example, you must place your bet at a roulette wheel before the wheel is spun. Or you must decide what type of auto insurance to purchase before you find out whether you will be in an accident. However, before you make a decision, you must at least list the possible outcomes that might occur. In some cases, the outcomes will be a small set of discrete possibilities, such as the 11 possible sums (2 through 12) of the roll of two dice. In other cases, the outcomes will be a continuum of possibilities, such as the possible damage amounts to a car in an accident. In this chapter, we generally allow only a small discrete set of possible outcomes. If the actual set of outcomes is a continuum, we typically choose a small set of representative outcomes from this continuum.

Probabilities of Outcomes A list of all possible outcomes is not enough. As a decision maker, you must also assess the likelihoods of these outcomes with probabilities. These outcomes are generally not equally likely. For example, if there are only two possible outcomes, rain or no rain, when you are deciding whether to carry an umbrella to work, there is no generally no reason to assume that each of these outcomes has a 50-50 chance of occurring. Depending on the weather report, they might be 80-20, 30-70, or any of many other possibilities.

There is no easy way to assess the probabilities of the possible outcomes. Sometimes they will be determined at least partly by historical data. For example, if demand for your

1 For an interesting discussion of decision making at a very nontechnical level, we recommend the book by Hammond et al. (2015).

09953_ch06_ptg01_242-292.indd 245 04/03/19 6:13 PM

2 4 6 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

product is uncertain, with possible outcomes “low,” “medium,” and “high,” you might assess their probabilities as 0.5, 0.3, and 0.2 because past demands have been low about 50% of the time, medium about 30% of the time, and high about 20% of the time.2 How- ever, this product might be a totally new product, unlike any of your previous products. Then data on past demands will probably not be relevant, and your probability assess- ments for demand of the new product will necessarily contain a heavy subjective compo- nent—your best guesses based on your experience and possibly the inputs of the marketing experts in your company. In fact, probabilities in most real business decision-making problems are of the subjective variety, so managers must make the probability assessments most in line with the data available and their gut feeling.

To complicate matters, probabilities sometimes change as more information becomes available. For example, suppose you assess the probability that the Golden State Warriors will win the NBA championship this year. Will this assessment change if you hear later that Steph Curry has suffered a season-ending injury? It almost surely will, probably quite a lot. Sometimes, as in this basketball example, you will change your probabilities in an informal way when you get new information. However, in Section 6.6, we show how probabilities can be updated in a formal way by using an important law of probabilities called Bayes’ rule.

Payoffs and Costs Decisions and outcomes have consequences, either good or bad. These must be assessed before intelligent decisions can be made. In our problems, these will be monetary payoffs or costs, but in many real-world decision problems, they can be nonmonetary, such as environmental damage or loss of life. Nonmonetary consequences can be very difficult to quantify, but an attempt must be made to do so. Otherwise, it is impossible to make mean- ingful trade-offs.

Decision Criterion Once all of these elements of a decision problem have been specified, you must make some difficult trade-offs. For example, would you rather take a chance at receiving $1 million, with the risk of losing $2 million, or would you rather play it safer? Of course, the answer depends on the probabilities of these two outcomes, but as you will see later in the chapter, if very large amounts of money are at stake (relative to your wealth), your attitude toward risk can also play a key role in the decision-making process.

In any case, for each possible decision, you face a number of uncertain outcomes with given probabilities, and each of these leads to a payoff or a cost. The result is a probability distribution of payoffs and costs. For example, one decision might lead to the following: a payoff of $50,000 with probability 0.1, a payoff of $10,000 with probability 0.2, and a cost of $5000 with probability 0.7. (The three outcomes are mutually exclusive; their probabil- ities sum to 1.) Another decision might lead to the following: a payoff of $5000 with prob- ability 0.6 and a cost of $1000 with probability 0.4. Which of these two decisions do you favor? The choice is not obvious. The first decision has more upside potential but more downside risk, whereas the second decision is safer.

In situations like this—the same situations faced throughout this chapter—you need a decision criterion for choosing between two or more probability distributions of payoff/ cost outcomes. Several methods have been proposed:

• Look at the worst possible outcome for each decision and choose the decision that has the least bad of these. This is relevant for an extreme pessimist.

• Look at the 5th percentile of the distribution of outcomes for each decision and choose the decision that has the best of these. This is also relevant for a pessimist—or a com- pany that wants to limit its losses. (Any percentile, not just the 5th, could be chosen.)

2 As discussed in the previous chapter, there are several equivalent ways to express probabilities. As an example, you can state that the probability of your team winning a basketball game is 0.6. Alternatively, you can say that the probability of them winning is 60%, or that the odds of them winning are 3 to 2. These are all equivalent. We will generally express probabilities as decimal numbers between 0 and 1, but we will some- times quote percentages.

09953_ch06_ptg01_242-292.indd 246 04/03/19 6:13 PM

6-3 eMV and Decision trees 2 4 7

• Look at the best possible outcome for each decision and choose the decision that has the best of these. This is relevant for an extreme optimist.

• Look at the variance (or standard deviation) of the distribution of outcomes for each decision and choose the decision that has the smallest of these. This is relevant for mini- mizing risk but it treats upside risk and downside risk in the same way.

• Look at the downside risk (however you want to define it) of the distribution of outcomes for each decision and choose the decision with the smallest of these. Again, this is relevant for minimizing risk, but now it minimizes only the part of the risk you really want to avoid.

The point here is that a probability distribution of payoffs and costs has several summary measures that could be used a decision criterion, and you could make an argument for any of the measures just listed. However, the measure that has been used most often, and the one that will be used for most of this chapter, is the mean of the probability distribution, also called its expected value. Because we are dealing with monetary outcomes, this crite- rion is generally known as the expected monetary value, or EMV criterion. The EMV criterion has a long-standing tradition in decision-making analysis, both at a theoretical level (hundreds of scholarly journal articles) and at a practical level (used by many busi- nesses). It provides a rational way of making decisions, at least when the monetary pay- offs and costs are of “moderate” size relative to the decision maker’s wealth. (Section 6.7 presents another decision criterion when the monetary values are not “moderate.”)

The expected monetary value, or EMV, for any decision is a weighted average of the possible payoffs/costs for this decision, weighted by the probabilities of the outcomes. Using the EMV criterion, you choose the decision with the largest EMV. This is sometimes called “playing the averages.”

The EMV criterion is also easy to operationalize. For each decision, you take a weighted sum of the possible monetary outcomes, weighted by their probabilities, to find the EMV. Then you identify the largest of these EMVs. For the two decisions listed earlier, their EMVs are as follows:

• Decision 1: EMV 5 50000(0.1) 1 10000(0.3) 1 (25000)(0.6) 5 $3500

• Decision 2: EMV 5 5000(0.6) 1 (21000)(0.4) 5 $2600

Therefore, according to the EMV criterion, you should choose decision 1.

6-3 EMV and Decision Trees Because the EMV criterion plays such a crucial role in decision making under uncertainty, it is worth exploring in more detail.

First, if you are acting according to the EMV criterion, you value a decision with a given EMV the same as a sure monetary outcome with the same EMV. To see how this works, sup- pose there is a third decision in addition to the previous two. If you choose this decision, there is no risk at all; you receive a sure $3000. Should you make this decision, presumably to avoid risk? According to the EMV criterion, the answer is no. Decision 1, with an EMV of $3500, is equivalent (for an EMV maximizer) to a sure $3500 payoff. Hence, it is favored over the new riskless decision. (Read this paragraph several times and think about its consequences. It is sometimes difficult to accept this logic in real decision-making problems, which is why not everyone uses the EMV criterion in every situation.)

Second, the EMV criterion doesn’t guarantee good outcomes. Indeed, no criterion can guarantee good outcomes. If you make decision 1, for example, you might get lucky and make $50,000, but there is a 70% chance that you will lose $5000. This is the very nature of decision making under uncertainty: you make a decision and then you wait to see the consequences. They might be good and they might be bad, but at least by using the EMV criterion, you know that you have proceeded rationally.

09953_ch06_ptg01_242-292.indd 247 04/03/19 6:13 PM

2 4 8 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

Third, the EMV criterion is easy to operationalize in a spreadsheet. This is shown in Figure 6.1. (See the file Simple Decision Problem Finished.xlsx.) For any decision, you list the possible payoff/cost values and their probabilities. Then you calculate the EMV with a SUMPRODUCT function. For example, the formula in cell B7 is

5SUMPRODUCT(A3:A5,B3:B5)

Figure 6.1 EMV Calculations in Excel

1 2 3 4 5 6 7

A B C D E F G H Decision 1

Payoff/Cost $50,000 $10,000 –$5,000

EMV $3,500 EMV$2,600EMV $3,000

0.1 $5,000 −$1,000

0.6 $3,000 1 0.40.2

0.7

Probability Payoff/Cost Probability Probability Decision 2

Payoff/Cost Decision 3

The advantage to calculating EMVs in a spreadsheet is that you can easily perform sensitivity analysis on any of the inputs. For example, Figure 6.2 shows what happens when the good outcome for decision 2 becomes more probable (and the bad outcome becomes less probable). Now the EMV for decision 2 is the largest of the three EMVs, so it is the best decision.

Usually, the most important information from a sensitivity analysis is whether the best decision continues to be best as one or more inputs change.

Figure 6.2 EMV Calculations with Different Inputs

1 2 3 4 5 6 7

A B C D E F G H Decision 1

Payoff/Cost $50,000 $10,000 –$5,000

EMV $3,500 EMV$3,800EMV $3,000

0.1 $5,000 –$1,000

0.8 $3,000 1 0.20.2

0.7

Probability Payoff/Cost Probability Payoff/Cost Probability Decision 2 Decision 3

You might still be wondering why we choose the EMV criterion in the first place. One way of answering this is that EMV represents a long-run average. If—and this is a big if—the decision could be repeated many times, all with the same monetary values and probabilities, the EMV is the long-run average of the outcomes you would observe. For example, by making decision 1, you would gain $50,000 about 10% of the time, you would gain $10,000 about 20% of the time, and you would lose $5000 about 70% of the time. In the long run, your average net gain would be about $3500.

This argument might or might not be relevant. For a company that routinely makes many decisions of this type, even though they are not identical, long-term averages make sense. Sometimes they win, and sometimes they lose, but it makes sense for them to be concerned only with long-term averages. However, a particular decision problem is often a “one-shot deal.” It won’t be repeated many times in the future; in fact, it won’t be repeated at all. In this case, you might argue that a long-term average criterion makes no sense and that some other criterion should be used instead. This has been debated by decision analysts, including many academics, for years, and the arguments continue. Nevertheless, most analysts agree that when “moderate” amounts of money are at stake, the EMV crite- rion provides a rational way of making decisions, even for one-shot deals. Therefore, we use the EMV criterion in most of this chapter.

A decision problem evolves through time. A decision is made, then an uncertain out- come is observed, then another decision might need to be made, then another uncertain outcome might be observed, and so on. All the while, payoffs are being received or costs are being incurred. It is useful to show all these elements of the decision problem, includ- ing the timing, in a type of graph called a decision tree. A decision tree not only allows everyone involved to see the elements of the decision problem in an intuitive format, but it also provides a straightforward way of making the necessary EMV calculations.

09953_ch06_ptg01_242-292.indd 248 04/03/19 6:13 PM

6-3 eMV and Decision trees 2 4 9

The decision tree for the simple decision problem discussed earlier appears in Figure 6.3.

What It Means to Be an eMV Maximizer

An EMV maximizer, by definition, is indifferent when faced with the choice between entering a gamble with a given EMV and receiving a sure dollar amount in the amount of the EMV. For example, consider a gamble where you flip a fair coin and win $0 or $1000 depending on whether you get a head or a tail. If you are an EMV maximizer, you are indifferent between entering this gamble, which has EMV $500, and receiving $500 for sure. Similarly, if the gamble is between losing $1000 and winning $500, based on the flip of the coin, and you are an EMV maximizer, you are indifferent between entering this gamble, which has EMV 2$250, and pay- ing a sure $250 to avoid the gamble. (This latter scenario is the basis of insurance.)

Fundamental Insight

Figure 6.3 Simple Decision Tree

3500

50000

10000

�5000

5000

�1000

0.4

0.6

0.7

0.2

0.1

26003500

Decision 1

Decision 2

Decision 3

3000

This decision tree was actually created in Excel by using its built-in shape tools on a blank worksheet, but you could just as well draw it on a piece of paper. Alternatively, you could use the Palisade PrecisionTree add-in that we discuss later in the chapter. The important thing for now is how you interpret this decision tree. It is important to realize that decision trees such as this one have been used for over 50 years. They all use the fol- lowing basic conventions:

Decision Tree Conventions

1. Decision trees are composed of nodes (circles, squares, and triangles) and branches (lines).

2. The nodes represent points in time. A decision node (a square) represents a time when you make a decision. A probability node (a circle) represents a time when the result of an uncertain outcome becomes known. An end node (a triangle) indicates that the problem is completed—all decisions have been made, all uncertainty has been resolved, and all payoffs and costs have been incurred. (When people draw decision trees by hand, they often omit the actual triangles, as we have done in Figure 6.3. However, we still refer to the right-hand tips of the branches as the end nodes.)

09953_ch06_ptg01_242-292.indd 249 04/03/19 6:13 PM

2 5 0 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

The decision tree in Figure 6.3 follows these conventions. The decision node comes first (to the left) because you must make a decision before observing any uncertain out- comes. The probability nodes then follow the decision branches, and the probabilities appear above their branches. (Actually, there is no need for a probability node after deci- sion 3 branch because its monetary value is a sure $3000.) The ultimate payoffs or costs appear next to the end nodes, to the right of the probability branches. The EMVs above the probability nodes are for the various decisions. For example, the EMV for the decision 1 branch is $3500. The maximum of the EMVs corresponds to the decision 1 branch, and this maximum is written above the decision node. Because it corresponds to decision 1, we put a notch on the decision 1 branch to indicate that this decision is best.

This decision tree is almost a direct translation of the spreadsheet model in Figure 6.1. Indeed, a decision tree is overkill for such a simple problem; the spreadsheet model provides all of the required information. However, as you will see later, especially in Section 6.6, decision trees provide a useful view of more complex problems. In addition, decision trees provides a framework for doing all of the EMV calculations. They allow you to use the following folding-back procedure to find the EMVs and the best decision.

3. Time proceeds from left to right. This means that any branches leading into a node (from the left) have already occurred. Any branches leading out of a node (to the right) have not yet occurred.

4. Branches leading out of a decision node represent the possible decisions; you get to choose the branch you prefer. Branches leading out of probability nodes represent the possible uncertain outcomes; you have no control over which of these will occur.

5. Probabilities are listed on probability branches. These probabilities are conditional on the events that have already been observed (those to the left). Also, the probabilities on branches leading out of any probability node must sum to 1.

6. Monetary values are shown to the right of the end nodes. (As we discuss shortly, some monetary values can also be placed under the branches where they occur in time.)

7. EMVs are calculated through a “folding-back” process, discussed next. They are shown above the various nodes. It is then customary to mark the optimal decision branch(es) in some way. We have marked ours with a small notch.

Folding-Back Procedure

Starting from the right of the decision tree and working back to the left:

1. At each probability node, calculate an EMV—a sum of products of monetary values and probabilities.

2. At each decision node, take a maximum of EMVs to identify the optimal decision.3

3 Some decision problems involve only costs. In that case it is more convenient to label the tree with positive costs and take minimums of expected costs at the decision nodes.

This is exactly what we did in Figure 6.3. At each probability node, we calculated EMVs in the usual way (sums of products) and wrote them above the nodes. Then at the decision node, we took the maximum of the three EMVs and wrote it above this node.

09953_ch06_ptg01_242-292.indd 250 04/03/19 6:13 PM

6-4 One-Stage Decision problems 2 5 1

Although this procedure requires more work for more complex decision trees, the same two steps—taking EMVs at probability nodes and taking maximums at decision nodes— are the only arithmetic operations required. In addition, the PrecisionTree add-in discussed later in the chapter performs the folding-back calculations for you.

The folding-back process is a systematic way of calcu- lating EMVs in a decision tree and thereby identifying the best decision strategy.

b. Let the probability of the worst outcome for the first decision, the value in cell B5, vary from 0.7 to 0.9 in increments of 0.025, and use formulas in cells B3 and B4 to ensure that they remain in the ratio 1 to 2 and the three probabilities for decision 1 continue to sum to 1.

c. Use a two-way data table to let the inputs in parts a and b vary simultaneously over the indicated ranges.

Level B 3. Some decision makers prefer decisions with low risk, but

this depends on how risk is measured. As we mentioned in this section, variance (see the definition in problem 1) is one measure of risk, but it includes both upside and downside risk. That is, an outcome with a large positive payoff contributes to variance, but this type of “risk” is good. Consider a decision with some possible payoffs and some possible costs, with given probabilities. How might you develop a measure of downside risk for such a decision? With your downside measure of risk, which decision in Figure 6.1 do you prefer, decision 1 or deci- sion 2? (There is no single correct answer.)

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 1. Several decision criteria besides EMV are suggested in

the section. For each of the following criteria, rank all three decisions in Figure 6.1 from best to worst. a. Look only at the worst possible outcome for each

decision. b. Look only at the best possible outcome for each

decision. c. Look at the variance of the distribution of outcomes

for each decision, which you want to be small. (The variance of a probability distribution is the weighted sum of squared differences from the mean, weighted by the probabilities.)

2. For the decision problem in Figure 6.1, use data tables to perform the following sensitivity analyses. The goal in each is to see whether decision 1 continues to have the largest EMV. In each part, provide a brief explanation of the results. a. Let the payoff from the best outcome, the value in cell

A3, vary from $30,000 to $50,000 in increments of $2500.

6-4 One-Stage Decision Problems Many decision problems are similar to the simple decision problem discussed in the previous section. You make a decision, then you wait to see an uncertain outcome, and a payoff is received or a cost is incurred. We refer to these as single-stage decision problems because you make only one decision, the one right now. They all unfold in essentially the same way, as indicated by the spreadsheet model in Figure 6.1 or the deci- sion tree in Figure 6.3. The following example is typical of one-stage decision problems. This example is used as a starting point for more complex examples in later sections.

EXAMPLE

6.1 NEW PRODUCT DECISIONS AT ACME The Acme Company must decide whether to market a new product. As in many new-product situations, there is considerable uncertainty about the eventual success of the product. The product is currently part way through the development process, and some fixed development costs have already been incurred. If the company decides to continue development and then market the product, there will be additional fixed costs, and they are estimated to be $6 million. If the product is marketed, its unit margin (selling price minus variable cost) will be $18. Acme classifies the possible market results as “great,” “fair,” and “awful,” and it estimates the probabilities of these outcomes to be 0.45, 0.35, and 0.20, respectively. Finally, the company

09953_ch06_ptg01_242-292.indd 251 04/03/19 6:13 PM

As before, a decision tree is probably overkill for this problem, but it is shown in Figure 6.5. (All monetary and sales volumes are shown in thousands.) This tree indicates one of at least two equivalent ways to show the EMV calculations. The values at the end nodes ignore the fixed cost, which is instead shown under the decision branch as a negative number. There- fore, the 7074 value above the probability node is the expected net revenue, not including the fixed cost. Then the fixed cost is subtracted from this to obtain the 1074 value above the decision node.

Figure 6.6 shows an equivalent tree, where the fixed cost is still shown under the decision branch but is subtracted from each end node. Now the EMV above the probability node is after subtraction of the fixed cost. The two trees are equivalent and either is perfectly acceptable. However, the second tree provides the insight that two of the three outcomes result in a net loss to Acme, even though the weighted average, the EMV, is well in the positive range. (Besides, as you will see in the next section, the second tree is the way the PrecisionTree add-in does it.)

estimates that the corresponding sales volumes (in thousands of units sold) from these three outcomes are 600, 300, and 90, respectively. Assuming that Acme is an EMV maximizer, should it finish development and then market the product, or should it stop development at this point and abandon the product?4

Objective To use the EMV criterion to help Acme decide whether to go ahead with the product.

Where Do the Numbers Come From? Acme’s cost accountants should be able to estimate the monetary inputs: the fixed costs and the unit margin. (Any fixed costs already incurred are sunk and therefore have no relevance to the current decision.) The uncertain sales volume is really a con- tinuous variable but, as in many decision problems, Acme has replaced the continuum by three representative possibilities. The assessment of the probabilities and the sales volumes for these three possibilities might be based partly on historical data and market research, but they almost surely have a subjective component.

Solution The elements of the decision problem appear in Figure 6.4. (See the files New Product Decisions - Single-Stage 1a Finished.xlsx and New Product Decisions - Single-Stage 1b Finished.xlsx.) If the company decides to stop development and abandon the product, there are no payoffs, costs, or uncertainties; the EMV is $0. (Actually, this isn’t really an expected value; it is a sure $0.) On the other hand, if the company proceeds with the product, it incurs the fixed cost and receives $18 for every unit it sells. The probability distribution of sales volume given in the problem statement appears in columns A to C, and each sales volume is multiplied by the unit margin to obtain the net revenues in column D. Finally, the formula for the EMV in cell B12 is

5SUMPRODUCT(D8:D10,B8:B10)-B4

Because this EMV is positive, slightly over $1 million, the company is better off marketing the product than abandoning it.

Figure 6.4 Spreadsheet Model for Single-Stage New Product Decision

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16

A B C D E F G

Decision 1: Continue development and market the new product

Acme single-stage new product decision

Fixed cost Unit margin

% decrease in all sales volumes EMV for decision 1

Sensitivity analysis to percentage decrease in all sales volumes

5% 10% 15% 20%

$1,074,000 $720,300 $366,600

$12,900 –$340,800

Market Great Fair Awful

EMV

$6,000,000 $18

0% $1,074,000

% decrease EMV for decision 1Probability 0.45 0.35 0.20

$1,074,000

Sales volume 600,000 300,000

90,000

Net revenue $10,800,000

$5,400,000 $1,620,000

Decision 2: Stop development and abandon product No payoffs, no costs, no uncertainty EMV

2 5 2 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

4 To keep the model simple, we ignore taxes and the time value of money.

09953_ch06_ptg01_242-292.indd 252 04/03/19 6:13 PM

Then a data table is used in the usual way, with cell G3 as the column input cell, to calculate the EMV for various percentage decreases. As you can see, the EMV stays positive, so that marketing remains best, for decreases up to 15%. But if the decrease is 20%, the EMV becomes negative, meaning that the best decision is to abandon the product. In this case, the possible gains from marketing are not large enough to offset the fixed cost.

Figure 6.5 Decision Tree for New Product Model

7074

600(18) = 10800

300(18) = 5400

90(18) = 1620

Awful 0.20

Fair 0.35

Great 0.45

1074

Market product –6000

Abandon product

Figure 6.6 Equivalent Decision Tree

1074

600(18) – 6000 = 4800

300(18) – 6000 = –600

90(18) – 6000 = –4380

Awful 0.20

Fair 0.35

Great 0.45

1074

Market product –6000

Abandon product

Figure 6.7 Sensitivity Analysis 3 4 5 6 7 8 9

10 11 12

F G % decrease in all sales volumes

Sensitivity analysis to percentage decrease in all sales volumes

EMV for decision 1

% decrease

0% $1,074,000

$1,074,000 $720,300 $366,600

$12,900 –$340,800

5% 10% 15% 20%

EMV for decision 1

6-4 One-Stage Decision problems 2 5 3

Using the spreadsheet model in Figure 6.4, it is easy to perform a sensitivity analysis. Usually, the main purpose of such an analysis is to see whether the best decision changes as one or more inputs change. As an example, we will see whether the best decision continues to be “proceed with marketing” if the total market decreases. Specifically, we let each of the potential sales volumes decrease by the same percentage and we keep track of the EMV from marketing the product. The results appear in Figure 6.7. For any percentage decrease in cell G3, the EMV from marketing is calculated in cell G4 with the formula

5(1-G3)*SUMPRODUCT(D8:D10,B8:B10)-B4

09953_ch06_ptg01_242-292.indd 253 04/03/19 6:13 PM

2 5 4 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

The Acme problem is a prototype for all single-stage decision problems. When only a single decision needs to be made, and all of the elements of the decision problem have been specified, it is easy to calculate the required EMVs for the possible decisions and hence determine the EMV-maximizing decision in a spreadsheet model. The problem and the calculations can also be shown in a decision tree, although this doesn’t really provide any new information except possibly to give everyone involved a better “picture” of the decision problem. In the next section, we examine a multistage version of the Acme problem, and then the real advantage of decision trees will become evident.

Level B 7. Sometimes a “single-stage” decision can be broken

down into a sequence of decisions, with no uncertainty resolved between these decisions. Similarly, uncer- tainty can sometimes be broken down into a sequence of uncertain outcomes. Here is a typical example. A com- pany has a chance to bid on a government project. The company first decides whether to place a bid, and then if it decides to place a bid, it decides how much to bid. Once these decisions have been made, the uncertainty is resolved. First, the company observes whether there are any competing bids. Second, if there is at least one com- peting bid, the company observes the lowest competing bid. The lowest of all bids wins the contract. Draw a decision tree that reflects this sequence. There should be two “stages” of decision nodes, followed by two “stages” of probability nodes. Then label the tree with some reasonable monetary values and probabilities, and perform the folding back process to find the company’s best strategy. Note that if the company wins the contract, its payoff is its bid amount minus its cost of complet- ing the project minus its cost of preparing the bid, where these costs are assumed to be known.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 4. The fixed cost of $6 million in the Acme problem is

evidently not large enough to make Acme abandon the product at the current time. How large would the fixed cost need to be to make the abandon option the best option? Explain how the decision tree, especially the version in Figure 6.5, answers this question easily.

5. Perform a sensitivity analysis on the probability of a great market. To do this, enter formulas in cells B9 and B10 (see Figure 6.4) to ensure that the probabilities of “fair” and “awful” remain in the same ratio, 35 to 20, and that all three probabilities continue to sum to 1. Then let the prob- ability of “great” vary from 0.25 to 0.50 in increments of 0.05. Is it ever best to abandon the product in this range?

6. Sometimes it is possible for a company to influence the uncertain outcomes in a favorable direction. Suppose Acme could, by an early marketing blitz, change the prob- abilities of “great,” “fair,” and “awful” from their current values to 0.75, 0.15, and 0.10. In terms of EMV, how much would the company be willing to pay for such a blitz?

6-5 The PrecisionTree Add-In Decision trees present a challenge for Excel. The challenge is to take advantage of Excel’s calculation capabilities (to calculate EMVs, for example) and its graphical capabilities (to draw the decision tree). Using only Excel’s built-in tools, this is virtually impossible (or at least very painful) to do. Fortunately, Palisade has developed an Excel add-in called PrecisionTree that makes the process relatively straightforward. This add-in not only enables you to draw and label a decision tree, but it also performs the folding-back procedure automatically and then allows you to perform sensitivity analysis on key input parameters.

The first thing you must do to use PrecisionTree is to “add it in.” We assume you have already installed the Palisade DecisionTools Suite. Then to run PrecisionTree, you have two options:

• If Excel is not currently running, you can open Excel and PrecisionTree by selecting PrecisionTree from the Palisade group in the list of programs on your computer.

• If Excel is currently running, the first option will open PrecisionTree on top of Excel.

In either case, you will see the Welcome screen in Figure 6.8. Note the Quick Start link. We will come back to this shortly.

Once you click OK to dismiss the Welcome screen, you will know that PrecisionTree is loaded because of the new PrecisionTree tab and associated ribbon shown in Figure 6.9.

09953_ch06_ptg01_242-292.indd 254 04/03/19 6:13 PM

6-5 the precisiontree add-In 2 5 5

Although PrecisionTree is quite easy to use once you are familiar with it, you have to learn the basics. The easiest way to do this is to run a series of Quick Start videos. To do this, you can bring up the Welcome screen in Figure 6.8 at any time through the Precision- Tree Help dropdown list. Then you can click the Quick Start link on the Welcome screen. This opens an example file shown in Figure 6.10. The five buttons on the left each launch a video that explains the basic features of PrecisionTree. Rather than repeat this information here, we urge you to watch the videos and practice the steps—as often as you like. From here on, we assume that you have done so.

Figure 6.8 PrecisionTree Welcome Screen

Figure 6.9 PrecisionTree Ribbon

Figure 6.10 PrecisionTree Quick Start Buttons

1 2 3 4

$7,500 $150,000

75%

20% 40% 30% 10%

5 6 7 8 9

10 11 12 13 14 15

18 19

17 16

A B C D E F G H I J Bidding for a government contract

Known inputs Cost of placing a bid Cost of completing project Probability of any competing bid(s)

Probability distribution of low competitor bid (if any) Assuming at least one competitor bid... Value Probability Our bid

$160,000 $170,000 $180,000

80% 40% 10%

Probability of winning Less than $160,000 Between $160,000 and $170,000 Between $170,000 and $180,000 Greater than $180,000

click the following buttons in the order shown to see videos of the steps in the analysis:

Step 2 Build Skeleton

of Tree

Step 3 Enter Values and

Probabilities

Step 4 Examine Optimal

Strategy

Step 5 Perform Sensitivity

Analysis

More PrecisionTree Examples

Read PDF Instructions

Step 1 Plan Decision Tree

Model

09953_ch06_ptg01_242-292.indd 255 04/03/19 6:13 PM

2 5 6 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

It is instructive to examine PrecisionTree’s decision tree for Acme’s single stage prob- lem. The completed tree appears in Figure 6.11. (See the file New Product Decisions - Single-Stage - 1c Finished.xlsx.) It is essentially a mixture of the trees in Figures 6.4 and 6.5, and it is equivalent to each of them. As in Figure 6.4, the fixed cost is entered as a negative number below the decision branch, and the net revenues are entered below the probability branches. Then PrecisionTree calculates the net value—the sum of the monetary values on any path through the tree—to the right of the correspond- ing triangle end nodes. For the folding back process, it uses these net values. Specifi- cally, the 1074 value to the right of the probability node is calculated (automatically) as (4800)(0.45) 1 (2600)(0.35) 1 (24380)(0.20).5 Then the 1074 value to the right of the decision node is calculated as the maximum of 1074 and 0.

In other words, PrecisionTree draws essentially the same tree and makes the same calculations that you could do by hand. Its advantages are that (1) it generates a nice-looking tree with all of the relevant inputs displayed, (2) it performs the folding-back calculations automatically, and (3) it permits quick sensitivity analyses on any of the model inputs. Also, you can easily identify the best decisions by following the TRUE branches. We will continue to use PrecisionTree in the rest of the chapter for trees that are considerably more complex than the one in Figure 6.11.

single-stage_decision_ tree video.

Formatting Numbers

If you are careful about formatting numbers in Excel, you might spend a lot of time formatting all of the numbers in a decision tree just the way you like them. However, there is a much quicker way in PrecisionTree. From the Settings dropdown on the PrecisionTree ribbon, select Model Settings and then the Format tab. By entering the formats you prefer here, the entire tree is formatted appropriately.

PrecisionTree Tip

Figure 6.11 Decision Tree from PrecisionTree

1 Acme single-stage new product decision

Inputs

Fixed cost �$6,000,000

$18

A B C D

Unit margin

Market Probability

0.45

0.35

0.20

Sales volume

600,000

300,000

90,000

Net revenue

$10,800,000

$5,400,000

$1,620,000

45.0%

$4,800,000

35.0%

�$600,000

�$4,380,000

Great

Fair

Awful

21 New Product Decision

Continue with product? $1,074,000

�$6,000,000 TRUE Sales volume

$1,074,000

45.0%

$10,800,000

35.0% $5,400,000

20.0% $1,620,000

FALSE

0.0%

The best decision is to continue with the product. Its EMV is $1,074,000.

Great

Fair

Awful

Yes

5 PrecisionTree refers to the “probability” nodes and branches as “chance” nodes and branches. The terms are equivalent.

09953_ch06_ptg01_242-292.indd 256 04/03/19 6:13 PM

6-6 Multistage Decision problems 2 5 7

We finish this section with one important reminder discussed in the Quick Start vid- eos. PrecisionTree reserves the cells with colored font (green, red, and blue) for its special formulas, so you should not change these cells. Your entries—probabilities and monetary values—should all be in the cells with black font, and it is a good practice to cell reference these inputs whenever possible. For example, we didn’t enter 45% in cell C12; we entered a link to cell B8.

9. Use PrecisionTree’s Sensitivity Analysis tools to per- form the sensitivity analysis requested in problem 5 of the previous section. (Watch the Step 5 video in Figure 6.10 if necessary.)

Level B 10. Use PrecisionTree to solve problem 7 of the previous

section.

Problems Solutions for problems whose numbers appear within a colored box can be found in the Student Solution Files.

Level A 8. Explain in some detail how the PrecisionTree calculations

in Figure 6.11 for the Acme problem are exactly the same as those for the hand-drawn decision tree in Figure 6.6. In other words, explain exactly how PrecisionTree gets the monetary values in the colored cells in Figure 6.11.

6-6 Multistage Decision Problems Many real-world decision problems evolve through time in stages. A company first makes a decision. Then it observes an uncertain outcome that provides some infor- mation. Based on this information, the company then makes another decision. Then it observes another uncertain outcome. This process could continue for more stages, but we will limit the number of stages to two: a first decision, a first uncertain outcome, a second decision, and a second uncertain outcome. As time unfolds, payoffs are received and costs are incurred, depending on the decisions made and the uncertain outcomes observed. The objective is again to maximize EMV, but now we are searching for an EMV-maximizing strategy, often called a contingency plan, that specifies which deci- sion to make at each stage.

As you will see shortly, a contingency plan tells the company which decision to make at the first stage, but the company won’t know which decision to make at the second stage until the information from the first uncertain outcome is known. For example, if the infor- mation is bad news about a product, then the company might decide at the second stage to abandon the product, but if the news is good, the company might decide to continue with the product. This is the essence of a contingency plan: it specifies what do for each possi- ble uncertain outcome.

An important aspect of multistage decision problems is that probabilities can change through time. After you receive the information from the first-stage uncertain outcome, you might need to reassess the probabilities of future uncertain outcomes. As an example, if a new product is observed to do very poorly in a regional test market, your assessment of the probability that it will do well in a national market will almost surely decrease. Some- times this reassessment of probabilities can be done in an informal subjective manner. But whenever possible, it should be done with a probability law called Bayes’ rule. This rule provides a mathematical way of updating probabilities as new information becomes avail- able. We explain how it works in this section.

Another important aspect of multistage decision problems is the value of informa- tion. Sometimes the first-stage decision is to buy information that will help in making the second-stage decision. The question then is how much this information is worth. If you knew what the information would be, there would be no point in buying it. However, you

09953_ch06_ptg01_242-292.indd 257 04/03/19 6:13 PM

2 5 8 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

virtually never know what the information will be; you can only assess the probabilities of various information outcomes. In such cases, the goal is to calculate the expected value of the information—how much better you would be with the information than without it—and then compare this to the actual cost of buying the information to see whether it is worth buying. Again, we explain how it works in this section.

We now show one way the Acme decision problem can be extended to two stages. Later in this section, we examine another multistage version of Acme’s problem.

EXAMPLE

6.2 NEW PRODUCT DECISIONS AT ACME WITH TECHNOLOGICAL UNCERTAINTY

In this version of the example, we assume as before that the new product is still in the development stage. However, we now assume that there is a chance that the product will be a failure for technological reasons, such as a new drug that fails to meet FDA approval. At this point in the development process, Acme assesses the probability of technological failure to be 0.2. The $6 million fixed cost from before is now broken down into two components: $4 million for addition development costs and $2 million for fixed costs of marketing, the latter to be incurred only if the product is a technological success and the company decides to market it. The unit margin and the probability distribution of the product’s sales volume if it is marketed are the same as before. How should Acme proceed?

Objective To use a decision tree to find Acme’s EMV-maximizing strategy for this two-stage decision problem.

Where Do the Numbers Come From? The probability of technological failure might be based partly on historical data—the technological failure rate of similar prod- ucts in the past—but it is probably partly subjective, based on how the product’s development has proceeded so far. The prob- ability distribution of sales volume is a more difficult issue. When Acme makes its first decision, right now, it must look ahead to see how the market might look in the future, after the development stage, which could be quite a while from now. (The same issue is relevant in Example 6.1, although we didn’t discuss it there.) This a difficult assessment, and it is an obvious candidate for an eventual sensitivity analysis.6

Solution The reason this is a two-stage decision problem is that Acme can decide right away to stop development and abandon the prod- uct, thus saving further fixed costs of development. However, if Acme decides to continue development and the product turns out to be a technological success, a second decision on whether to market the product must still be made.

A spreadsheet model such as in Figure 6.1 for the single-stage problem could be developed to calculate the relevant EMVs, but this isn’t as easy as it sounds. A much better way is to use a decision tree, using the PrecisionTree add-in. The fin- ished tree appears in Figure 6.12. (See the file New Product Decisions - Technological Uncertainty Finished.xlsx.) The first decision is whether to continue development. If “Yes,” the fixed development cost is incurred, so it is entered on this branch. Then there is a probability node for the technological success or failure. If it’s a failure, there are no further costs, but the fixed development cost is lost. If it’s a success, Acme must decide whether to market the product. From this point, the tree is exactly like the single-stage tree, except that the fixed development cost has been incurred.

By following the TRUE branches, you can see Acme’s best strategy. The company should continue development, and if the product is a technological success, it should be marketed. The EMV, again the weighted average of all possible monetary outcomes with this strategy, is $59,200. However, this is only the expected value, or mean, of the probability distribution of monetary outcomes. You can see the full probability distribution by requesting a risk profile from PrecisionTree (through the Decision Analysis dropdown). This appears, both in graphical and tabular form, in Figure 6.13. Note that Acme has a 64% chance of incurring a net loss with this strategy, including a possible loss of $4.38 million. This doesn’t sound good. However, the company has a 36% chance of a net gain of $4.8 million and, in an expected value sense, this more than offsets the possi- ble losses.

6 We have purposely avoided the use of Bayes’ rule for now. It will be used in the next version of the Acme problem.

09953_ch06_ptg01_242-292.indd 258 04/03/19 6:13 PM

Figure 6.13 Risk Profile from Best Strategy

Figure 6.12 Decision Tree with Possible Technological Failure

A 1 2 3 Inputs

Acme multistage new product decisions with technological uncertainty

Probability of technological failure Fixed development cost Fixed marketing cost Unit margin

Market Probability Sales volume Net revenue $10,800,000

$5,400,000 $1,620,000

Probability of technological success

Great Fair Awful

4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

FALSE

TRUE

�$4,000,000�$4,000,000

0.0%

20.0%

80.0%80.0%

20.0%

–4,000,000

Market product?

45.0%

$10,800,000

35.0%

$5,400,000

$1,620,000

20.0%

4,800,000

�600,000

16.0%

�4,380,000

0.0%

–4,000,000

36.0%

28.0%

FALSE

TRUE

�$2,000,000�$2,000,000

31 32 33 34

B C D E F

Continue development?

Sales volume

59,200

Yes

1,074,000

New Product Decisions

Fair

Great

Awful

1,074,000

Technological success?

59,200

The best strategy is to continue development and, if there is technological success, market the product. The EMV from this strategy is $59,200.

600,000 300,000

90,000

0.45

0.2 0.8

–$4,000,000 –$2,000,000

$18

0.35 0.20

Chart Data Optimal Path

Value �4,380,000#1

#2 #3 #4

16.0000% 20.0000% 28.0000% 36.0000%

�4,000,000 �600,000

4,800,000

Probability

Probabilities for Decision Tree ‘New Product Decisions’ Optimal Path of Entire Decision Tree

40%

35%

30%

25%

20%

15% +

10%

�5 ,0

00 ,0

Pr ob

ab ili

�4 ,0

00 ,0

�3 ,0

00 ,0

�2 ,0

00 ,0

�1 ,0

00 ,0

1, 00

0, 00

2, 00

0, 00

4, 00

0, 00

5, 00

0, 00

3, 00

0, 00

6-6 Multistage Decision problems 2 5 9

09953_ch06_ptg01_242-292.indd 259 04/03/19 6:13 PM

We won’t perform any systematic sensitivity analyses on this model (we ask you to do some in the problems), but it is easy to show that the best strategy is quite sensitive to the probability of technological success. If you change this probability from 0.8 to 0.75 in cell B4, the tree automatically recalculates, with the results in Figure 6.14. With just this small change, the best decision changes completely. Now the company should discontinue development and abandon the product. There is evidently not a large enough chance of recovering the fixed development cost.

Placement of Results

When you request a risk profile or other PrecisionTree reports, they are placed in a new workbook by default. If you would rather have them placed in the same workbook as your decision tree, select Application Settings from the Utilities dropdown list on the PrecisionTree ribbon, and change the “Place Reports In” setting to Active Workbook. You only have to do this once.

PrecisionTree Tip

A B C D E F 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

New Product Decisions

FALSE

�$4,000,000

Technological success?

�194,500

25.0%

75.0% Market product?

1,074,000

FALSE

TRUE Sales volume

1,074,000

45.0%

$10,800,000

35.0%

$5,400,000

20.0%

$1,620,000

�$2,000,000

0.0%

4,800,000

0.0%

�600,000

0.0%

�4,380,000

0.0%

�4,000,000

0.0%

�4,000,000

100.0%

TRUE

Continue development?

Yes

Great

Fair

Awful

Figure 6.14 Decision Tree with Larger Probability of Failure

Modeling Issues We return to the probability distribution of eventual sales volume. The interpretation here is that at the time of the first decision, Acme has assessed what the market might look like after the development stage, which could be quite a while from now. Again, this is a difficult assessment. Acme could instead break this assessment into parts. It could first assess a probability distribution for how the general market for such products might change—up, down, or no change, for example—by the time development is completed. Then for each of these general markets, it could assess a probability distribution for the sales volume of its new product. By breaking it up in this way, Acme might be able to make a more accurate assessment, but the decision tree would be somewhat more complex. We ask you to explore this in one of the problems.

2 6 0 C h a p t e r 6 D e c i s i o n M a k i n g U n d e r U n c e r t a i n t y

The next example illustrates another possible multistage extension of the Acme deci- sion problem. This example provides an opportunity to introduce two important topics discussed earlier: Bayes’ rule for updating probabilities and the value of information.

09953_ch06_ptg01_242-292.indd 260 04/03/19 6:13 PM

6-6 Multistage Decision problems 2 6 1

EXAMPLE

6.3 NEW PRODUCT DECISIONS AT ACME WITH AN OPTION TO BUY INFORMATION

Suppose now that Acme has just about finished the development process on the new product, so that fixed development costs are no longer an issue, and technological failure is no longer a possibility. The only question is whether Acme should mar- ket the product, given the uncertainty about the eventual sales volume. If the company decides to market the product, it will incur fixed marketing costs of $4 million. To keep the model simple, we now assume that there are only two possible market outcomes, good or bad. The sales volumes for these two possible outcomes are 600,000 units and 100,000 units, and Acme assesses that their probabilities are 0.4 and 0.6. However, before making the ultimate decision, Acme has the option to hire a well-respected marketing research firm for $150,000. If Acme decides to use this option, the result will be a prediction of good or bad. That is, the marketing research firm will predict that either “We think the market for this product will be good” or “We think the market for this product will be bad.” Acme has used this firm before, so it has a sense of the prediction accuracy, as indicated in Table 6.1. Each row in this table indicates the actual market outcome, and each column indicates the prediction. If the actual market is good, the prediction will be good with probability 0.8 and bad with probability 0.2. If the actual market is bad, the prediction will be bad with probability 0.7 and good with probability 0.3. What should Acme do to maximize its EMV?

actual/predicted Good Bad

Good 0.8 0.2

Bad 0.3 0.7

Table 6.1 Prediction Accuracy of Marketing Research Firm

Objective To use a decision tree to see whether the marketing research firm is worth its cost and whether the product should be marketed.

Where Do the Numbers Come From? The main question here concerns the probabilities. Acme’s assessment of the probabilities of good or bad markets, 0.4 and 0.6, would be assessed as in earlier examples, probably subjectively. The probabilities in Table 6.1 are probably based partly on historical dealings with the marketing research firm—perhaps they have been right in about 75% of their predictions, give or take a bit—and some subjectivity. In any case, all of these probabilities are prime candidates for sensitivity analysis.

Solution Acme must first decide whether to hire the marketing research firm. If it decides not to, it can then immediately decide whether to market the product. On the other hand, if it decides to hire the firm, it must then wait for the firm’s prediction. After the pre- diction is received, Acme can then make the ultimate decision on whether to market the product. However, when making this ultimate decision, Acme should definitely take the firm’s prediction into account.

A “skeleton” for the appropriate decision tree, without any of the correct probabilities or monetary val- ues, appears in Figure 6.15. For now, just focus on the structure of the tree, and how time flows from left to right. The tree is not symmetric. If Acme doesn’t hire the firm, this “No” decision branch is followed imme- diately by another decision node for whether to market the product. However, if Acme does hire the firm, this “Yes” decision branch is followed by a probability node for the firm’s prediction. Then after a good or a bad prediction, there is a decision node for whether to market the product.

Now it is time to label the tree with probabilities and monetary values. The monetary values present no problems, as will be seen shortly. However, the probabilities require a digression so that we can discuss Bayes’ rule. The problem is that the given probabilities in Table 6.1 are not the probabilities required in the tree. After we discuss Bayes’ rule, we will return to Acme’s decision problem.

multistage_ decision_tree video.

09953_ch06_ptg01_242-292.indd 261 04/03/19 6:13 PM

Figure 6.15 Skeleton of Decision Tree with Option to Buy Information

17 A B C D E F

18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

New Product Decisions

50.0%

Bad

50.0%

Yes

50.0%

25.0%

0.0%

25.0%

0.0%

Market product?

Prediction

Sales volume

Bad

Good

Yes

Good

Yes

Good

Bad

Good

Bad

TRUE

FALSE

Hire firm?