Activity 3 - Advanced Statistical Concepts and Business Analytics

voyage

bowerman_9e_chap_151.pptx

Home >Business & Finance homework help >Activity 3 - Advanced Statistical Concepts and Business Analytics

Chapter 15

Multiple Regression and Model Building

Chapter Outline

15.1 The Multiple Regression Model and the Least Squares Point Estimate

15.2 R2 and Adjusted R2

15.3 Model Assumptions and the Standard Error

15.4 The Overall F Test

15.5 Testing the Significance of an Independent Variable

15.6 Confidence and Prediction Intervals

15-2

Chapter Outline Continued

15.7 The Sales Representative Case: Evaluating Employee Performance

15.8 Using Dummy Variables to Model Qualitative Independent Variables (Optional)

15.9 Using Squared and Interaction Variables (Optional)

15.10 Multicollinearity, Model Building and Model Validation (Optional)

15.11 Residual Analysis and Outlier Detection in Multiple Regression (Optional)

15-3

15.1 The Multiple Regression Model and the Least Squares Point Estimate

Simple linear regression used one independent variable to explain the dependent variable

Some relationships are too complex to be described using a single independent variable

Multiple regression uses two or more independent variables to describe the dependent variable

This allows multiple regression models to handle more complex situations

There is no limit to the number of independent variables a model can use

Multiple regression has only one dependent variable

LO15-1: Explain the multiple regression model and the related least squares point estimates.

15-4

The Multiple Regression Model

The linear regression model relating y to x1, x2,…, xk is y = β0 + β1x1 + β2x2 +…+ βkxk + 

µy = β0 + β1x1 + β2x2 +…+ βkxk is the mean value of the dependent variable y when the values of the independent variables are x1, x2,…, xk

β0, β1, β2,… βk are the unknown regression parameters relating the mean value of y to x1, x2,…, xk

 is an error term that describes the effects on y of all factors other than the independent variables x1, x2,…, xk

LO15-1

15-5

The Least Squares Estimates and Point Estimation and Prediction

Estimation/prediction equation ŷ = b0 + b1x1 + b2x2 + … + bkxk

is the point estimate of the mean value of the dependent variable when the values of the independent variables are x1, x2,…, xk

It is also the point prediction of an individual value of the dependent variable when the values of the independent variables are x1, x2,…, xk

b0, b1, b2,…, bk are the least squares point estimates of the parameters β0, β1, β2,…, βk

x1, x2,…, xk are specified values of the independent predictor variables x1, x2,…, xk

LO15-1

15-6

LO15-1

Example 15.1 The Tasty Sub Shop Case

Figure 15.4 (a)

15-7

15.2 R2 and Adjusted R2

Total variation is given by the formula

Explained variation is given by the formula

Unexplained variation is given by the formula

Total variation is the sum of explained and unexplained variation

LO15-2: Calculate and interpret the multiple and adjusted multiple coefficients of determination.

15-8

R2 and Adjusted R2 Continued

The multiple coefficient of determination is the ratio of explained variation to total variation

R2 is the proportion of the total variation that is explained by the overall regression model

Multiple correlation coefficient R is the square root of R2

LO15-2

15-9

Multiple Correlation Coefficient R

The multiple correlation coefficient R is just the square root of R2

With simple linear regression, r would take on the sign of b1

There are multiple bi’s with multiple regression

For this reason, R is always positive

To interpret the direction of the relationship between the x’s and y, you must look to the sign of the appropriate bi coefficient

LO15-2

15-10

Adjusted R2

Adding an independent variable to multiple regression will raise R2

R2 will rise slightly even if the new variable has no relationship to y

corrects this tendency in R2

As a result, it gives a better estimate of the importance of the independent variables

LO15-2

15-11

15.3 Model Assumptions and the Standard Error

The model is y = β0 + β1x1 + β2x2 + … + βkxk + 

Assumptions are stated about the model error terms, ’s

Mean of Zero Assumption: The mean of the error terms is equal to 0

Constant Variance Assumption: The variance of the error terms σ2 is, the same for every combination values of x1, x2,…, xk

Normality Assumption: The error terms follow a normal distribution for every combination values of x1, x2,…, xk

Independence Assumption: The values of the error terms are statistically independent of each other

LO15-3: Explain the assumptions behind multiple regression and calculate the standard error.

15-12

The Mean Square Error and the Standard Error

Sum of squared errors

Mean squared error

Point estimate of the residual variance σ2

Standard error

Point estimate of the residual standard deviation σ

LO15-3

15-13

15.4 The Overall F Test

To test H0: β1= β2 = …= βk = 0 versus Ha: At least one of β1, β2,…, βk ≠ 0

Test statistic

Reject H0 in favor of Ha if F(model) > F* or p-value < 

*F is based on k numerator and n - (k + 1) denominator degrees of freedom

LO15-4: Test the significance of a multiple regression model by using an F test.

15-14

15.5 Testing the Significance of an Independent Variable

A variable in a multiple regression model is not likely to be useful unless there is a significant relationship between it and y

To test significance, we use the null hypothesis H0: βj = 0

Versus the alternative hypothesis Ha: βj ≠ 0

LO15-5: Test the significance of a single independent variable.

15-15

Testing the Significance of the Independent Variable xj

LO15-5

15-16

Testing the Significance of an Independent Variable Continued

Customary to test significance of every independent variable

If we can reject H0: βj = 0 at =0.05, we have strong evidence the independent variable xj is significantly related to y

If we can reject H0: βj = 0 at =0.01, we have very strong evidence the independent variable xj is significantly related to y

The smaller the significance level  at which H0 can be rejected, the stronger the evidence that xj is significantly related to y

LO15-5

15-17

A Confidence Interval for the Regression Parameter βj

If the regression assumptions hold, 100 (1 - ) percent confidence interval for βj is [bj ± t/2 Sbj]

t/2 is based on n – (k + 1) degrees of freedom

LO15-5

15-18

15.6 Confidence and Prediction Intervals

The point on the regression line corresponding to a particular value of x1, x2,…, xk, of the independent variables is

It is unlikely that this value will equal the mean value of y for these x values

Therefore, we need to place bounds on how far away the predicted value might be

We can do this by calculating a confidence interval for the mean value of y and a prediction interval for an individual value of y

LO15-6: Find and interpret a confidence interval for a mean value and a prediction interval for an individual value.

15-19

Distance Value

Both the confidence interval for the mean value of y and the prediction interval for an individual value of y employ a quantity called the distance value

With simple regression, we were able to calculate the distance value fairly easily

However, for multiple regression, calculating the distance value requires matrix algebra

LO15-6

15-20

A Confidence Interval and a Prediction Interval

Distance value

Assume the regression assumptions hold

Confidence interval for the mean value of y

Prediction interval for an individual value of y

These are based on n - (k + 1) degrees of freedom

LO15-6

15-21

15.7 The Sales Representative Case: Evaluating Employee Performance

yi Yearly sales of the company’s product

x1 Number of months the representative has been employed

x2 Sales of products in the sales territory

x3 Dollar advertising expenditure in the territory

x4 Weighted average of the company’s market share in territory for the previous four years

x5 Change in the company’s market share in the territory over the previous four years

15-22

Partial Excel Output of a Regression Analysis of the Sales Territory Performance Data

Figure 15.10a

15-23

Time = 85.42

MktPoten = 35,182.73

Adver = 7,281.65

MktShare = 9.64

Change = .28

Sales

Predicted 4,181.74

95% Prediction Interval

[3,233.59 to 5,129.89]

15.8 Using Dummy Variables to Model Qualitative Independent Variables (Optional)

So far, we have only looked at including quantitative data in a regression model

However, we may wish to include descriptive qualitative data as well

For example, might want to include the gender of respondents

We can model the effects of different levels of a qualitative variable by using what are called dummy variables

Also known as indicator variables

LO15-7: Use dummy variables to model qualitative independent

Variables (Optional).

15-24

Constructing Dummy Variables

A dummy variable always has a value of either 0 or 1

For example, to model sales at two locations, would code the first location as a zero and the second as a 1

Operationally, it does not matter which is coded 0 and which is coded 1

LO15-7

15-25

What If We Have More Than Two Categories?

Consider having three categories, say A, B and C

Cannot code this using one dummy variable

A=0, B=1 and C=2 would be invalid

Assumes the difference between A and B is the same as B and C

We must use multiple dummy variables

Specifically, k categories requires k - 1 dummy variables

LO15-7

15-26

What If We Have Three Categories?

For A, B, and C, would need two dummy variables

x1 is 1 for A, zero otherwise

x2 is 1 for B, zero otherwise

If x1 and x2 are zero, must be C

This is why the third dummy variable is not needed

LO15-7

15-27

Interaction Models

So far, have only considered dummy variables as stand-alone variables

Model so far is y = β0 + β1x + β2DM + 

Where D is dummy variable

However, can also look at interaction between dummy variable and other variables

That model would take the form y = β0 + β1x + β2DM + β3xDM + 

With an interaction term, both the intercept and slope are shifted

LO15-7

15-28

15.9 Using Squared and Interaction Variables (Optional)

Quadratic regression model is: y = β0 + β1x + β2x2 ε

where

β0 + β1x + β2x2 is μy

β, β1, and β2 are the regression parameters

ε is an error term

LO15-8: Use squared and interaction variables.

15-29

Using Interaction Variables

Regression models often contain interaction variables

Formed by multiplying two independent variables together

Consider a model where x3 and x4 interact and x3 is used as a quadratic y = β0 + β1x4 + β2x3 + β3x32 + β4x4x3 + ε

LO15-8

15-30

15.10 Multicollinearity, Model Building, and Model Validation (Optional)

Multicollinearity: when “independent” variables are related to one another

Considered severe when the simple correlation exceeds 0.9

Even moderate multicollinearity can be a problem

Another measurement is variance inflation factors

Multicollinearity considered

Severe when VIF > 10

Moderately strong for VIF > 5

LO15-9: Describe multicollinearity and build and validate a multiple regression model (Optional).

15-31

Effect of Adding Independent Variable

Adding any independent variable will increase R²

Even adding an unimportant independent variable

Thus, R² cannot tell us that adding an independent variable is undesirable

LO15-9

15-32

A Better Criterion is the Standard Error

A better criterion is the size of the standard error s

If s increases when an independent variable is added, we should not add that variable

However, decreasing s alone is not enough

An independent variable should only be included if it reduces s enough to offset the higher t value and reduces the length of the desired prediction interval for y

LO15-9

15-33

C Statistic

Another quantity for comparing regression models is called the C (a.k.a. Cp) statistic,

First, calculate mean square error for the model containing all p potential independent variables (s2p)

Next, calculate SSE for a reduced model with k independent variables

LO15-9

15-34

C Statistic Continued

We want the value of C to be small

Adding unimportant independent variables will raise the value of C

While we want C to be small, we also wish to find a model for which C roughly equals k + 1

A model with C substantially greater than k + 1 has substantial bias and is undesirable

If a model has a small value of C and C for this model is less than k + 1, then it is not biased and the model should be considered desirable

LO15-9

15-35

The Partial F Test: An F Test for a Portion of a Regression Model

To test

H0: All of the βj coefficients corresponding to the independent variables in the subset are zero

Ha: At least one of the βj coefficients is not equal to zero

Reject H0 in favor of Ha if:

F(partial) > F or

p-value < 

F is based on k - g numerator and n - (k + 1) denominator degrees of freedom

LO15-9

15-36

15.11 Residual Analysis and Outlier Detection in Multiple Regression (Optional)

For an observed value of yi, the residual is i = yi - ŷ = yi – (b0 + b1xi1 + … + bkxik)

If the assumptions hold, the residuals should look like a random sample from a normal distribution with mean 0 and variance σ2

Residual plots

Residuals versus each independent variable

Residuals versus predicted y’s

Residuals in time order (if the response is a time series)

LO15-10: Use residual analysis and outlier detection to check the assumptions of multiple regression (Optional).

15-37

Figure 15.35

LO15-10

Outliers

Figure 15.37 c, d and e

15-38