business statics final (2hour duration)

jimmy121

Chapter14.pdf

Home >Business & Finance homework help >business statics final (2hour duration)

Chapter 14 Simple Linear Regression

 Simple Linear Regression Model

 Least Squares Method

 Coefficient of Determination

 Model Assumptions

 Testing for Significance

 Using the Estimated Regression Equation

for Estimation and Prediction

 Computer Solution

 Residual Analysis: Validating Model Assumptions

Simple Linear Regression Model

Y = b0 + b1x +e

where:

b0 and b1 are called parameters of the model,

e is a random variable called the error term.

 The simple linear regression model is:

 The equation that describes how y is related to x and an error term is called the regression model.

Simple Linear Regression Equation

 The simple linear regression equation is:

• E(Y) is the expected value of Y for a given x value. • b1 is the slope of the regression line. • b0 is the y intercept of the regression line. • Graph of the regression equation is a straight line.

E(Y) = b0 + b1x

Simple Linear Regression Equation

 Positive Linear Relationship

E(Y)

Slope b1 is positive

Regression line

Intercept b0

Simple Linear Regression Equation

 Negative Linear Relationship

E(Y)

Slope b1 is negative

Regression lineIntercept b0

Simple Linear Regression Equation

 No Relationship

E(Y)

Slope b1 is 0

Regression line Intercept

Estimated Simple Linear Regression Equation

 The estimated simple linear regression equation

0 1ŷ b b x 

• is the estimated value of Y for a given x value.ŷ • b1 is the slope of the line. • b0 is the y intercept of the line. • The graph is called the estimated regression line.

Estimation Process

Regression Model Y = b0 + b1x +e

Regression Equation E(Y) = b0 + b1x

Unknown Parameters b0, b1

Sample Data: x y

x1 y1 . . . .

xn yn

b0 and b1 provide estimates of

b0 and b1

Estimated Regression Equation

Sample Statistics b0, b1

0 1ŷ b b x 

Least Squares Method

 Least Squares Criterion

min (y yi i  ) 2

where:

yi = observed value of the dependent variable

for the ith observation ^yi = estimated value of the dependent variable

for the ith observation

Least Squares Method

 Slope for the Estimated Regression Equation

1 2

( )( )

( )

i i

x x y y b

x x

  



 

 y-Intercept for the Estimated Regression Equation

Least Squares Method

0 1b y b x 

where: xi = value of independent variable for ith

observation

n = total number of observations

_ y = mean value for dependent variable

_ x = mean value for independent variable

yi = value of dependent variable for ith observation

Kako Auto periodically has

a special week-long sale.

As part of the advertising

campaign Kako runs one or

more television commercials

during the weekend preceding the sale. Data from a

sample of 5 previous sales are shown on the next slide.

Simple Linear Regression

 Example: Kako Auto Sales

Simple Linear Regression

 Example: Kako Auto Sales

Number of TV Ads

Number of Cars Sold

1 3 2 1 3

14 24 18 17 27

Estimated Regression Equation

ˆ 10 5y x 

1 2

( )( ) 20 5

( ) 4

i i

x x y y b

x x

    



 

0 1 20 5( 2 ) 10b y b x    

 Slope for the Estimated Regression Equation

 y-Intercept for the Estimated Regression Equation

 Estimated Regression Equation

Scatter Diagram and Trend Line

y = 5x + 10

0 1 2 3 4 TV Ads

C a

r s

S o

Coefficient of Determination

 Relationship Among SST, SSR, SSE

where:

SST = total sum of squares

SSR = sum of squares due to regression

SSE = sum of squares due to error

SST = SSR + SSE

2 ( )iy y

2ˆ( )iy y  2ˆ( )i iy y 

 The coefficient of determination is:

Coefficient of Determination

where:

SSR = sum of squares due to regression

SST = total sum of squares

r2 = SSR/SST

Coefficient of Determination

r2 = SSR/SST = 100/114 = 0.8772

The regression relationship is very strong; 88%

of the variability in the number of cars sold can be

explained by the linear relationship between the

number of TV ads and the number of cars sold.

Sample Correlation Coefficient

1 ) of(sign rbr

xy 

ionDeterminat oft Coefficien ) of(sign 1

br xy



where:

b1 = the slope of the estimated regression

equation xbby 10

ˆ 

1 ) of(sign rbr

xy 

The sign of b1 in the equation is “+”.ˆ 10 5y x 

= + .8772xyr

Sample Correlation Coefficient

rxy = +0.9366

Assumptions About the Error Term e

1. The error e is a random variable with mean of zero.

2. The variance of e , denoted by  2, is the same for all values of the independent variable.

3. The values of e are independent.

4. The error e is a normally distributed random variable.

Testing for Significance

To test for a significant regression relationship, we must conduct a hypothesis test to determine whether the value of b1 is zero.

Two tests are commonly used:

t Test and F Test

Both the t test and F test require an estimate of  2, the variance of e in the regression model.

Testing for Significance

 An Estimate of 

  2

2 )()ˆ(SSE

iiii xbbyyy

where:

s 2 = MSE = SSE/(n  2)

The mean square error (MSE) provides the estimate

of  2, and the notation s2 is also used.

Testing for Significance

 An Estimate of 

SSE MSE

 

n s

• To estimate  we take the square root of  2.

• The resulting s is called the standard error of the estimate.

Testing for Significance: t Test

 Hypotheses

 Test Statistic

0 1: 0H b 

1: 0aH b 

b t

s 

 Rejection Rule

Testing for Significance: t Test

where:

t is based on a t distribution

with n - 2 degrees of freedom

Reject H0 if p-value <  or t < -t or t > t

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

 = 0.05

4. State the rejection rule. Reject H0 if p-value < 0.05 or |t| > 3.182 (with

3 degrees of freedom)

Testing for Significance: t Test

0 1: 0H b 

1: 0aH b 

b t

s 

Testing for Significance: t Test

5. Compute the value of the test statistic.

6. Determine whether to reject H0.

t = 4.541 provides an area of .01 in the upper tail. Hence, the p-value is less than 0.02. (Also, t = 4.63 > 3.182.) We can reject H0.

1 5 4.63 1.08b

b t

s   

Confidence Interval for b1

 H0 is rejected if the hypothesized value of b1 is not included in the confidence interval for b1.

 We can use a 95% confidence interval for b1 to test the hypotheses just used in the t test.

Confidence Interval for b1

 The form of a confidence interval for b1 is:

11 / 2 bb t s

where is the t value providing an area

of /2 in the upper tail of a t distribution

with n - 2 degrees of freedom

2/ t

b1 is the point

estimator

is the margin of error

1/ 2 b t s



Confidence Interval for b1

Reject H0 if 0 is not included in

the confidence interval for b1.

0 is not included in the confidence interval.

Reject H0

= 5 +/- 3.182(1.08) = 5 +/- 3.44 12/1 b

stb 



or 1.56 to 8.44

 Rejection Rule

 95% Confidence Interval for b1

 Conclusion

 Hypotheses

 Test Statistic

Testing for Significance: F Test

F = MSR/MSE

0 1: 0H b 

1: 0aH b 

 Rejection Rule

Testing for Significance: F Test

where:

F is based on an F distribution with

1 degree of freedom in the numerator and

n - 2 degrees of freedom in the denominator

Reject H0 if p-value <  or F > F

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

 = 0.05

4. State the rejection rule. Reject H0 if p-value < 0.05 or F > 10.13 (with 1 d.f.

in numerator and 3 d.f. in denominator)

Testing for Significance: F Test

0 1: 0H b 

1: 0aH b 

F = MSR/MSE

Testing for Significance: F Test

5. Compute the value of the test statistic.

6. Determine whether to reject H0.

F = 17.44 provides an area of 0.025 in the upper tail. Thus, the p-value corresponding to F = 21.43 is less than 2(0.025) = 0.05. Hence, we reject H0.

F = MSR/MSE = 100/4.667 = 21.43

The statistical evidence is sufficient to conclude that we have a significant relationship between the number of TV ads aired and the number of cars sold.

Some Cautions about the Interpretation of Significance Tests

 Just because we are able to reject H0: b1 = 0 and demonstrate statistical significance does not enable us to conclude that there is a linear relationship between x and y.

 Rejecting H0: b1 = 0 and concluding that the relationship between x and y is significant does not enable us to conclude that a cause-and-effect relationship is present between x and y.

Using the Estimated Regression Equation for Estimation and Prediction

 / y t sp y p  2

where:

confidence coefficient is 1 -  and

t/2 is based on a t distribution

with n - 2 degrees of freedom

 Confidence Interval Estimate of E(yp)

 Prediction Interval Estimate of yp

Point Estimation

If 3 TV ads are run prior to a sale, we expect

the mean number of cars sold to be:

ŷ = 10 + 5(3) = 25 cars

 Excel’s Confidence Interval Output

D E F G

1 CONFIDENCE INTERVAL

2 x p 3

3 x bar 2.0

4 x p -x bar 1.0

5 (x p -x bar) 2

1.0

6  (x p -x bar) 2

4.0

7 Variance of y hat 2.1000

8 Std. Dev of y hat 1.4491

9 t Value 3.1824

10 Margin of Error 4.6118

11 Point Estimate 25.0

12 Lower Limit 20.39

13 Upper Limit 29.61

Confidence Interval for E(yp)

The 95% confidence interval estimate of the mean

number of cars sold when 3 TV ads are run is:

Confidence Interval for E(yp)

25 + 4.61 = 20.39 to 29.61 cars

 Excel’s Prediction Interval Output

H I

1 PREDICTION INTERVAL

2 Variance of yp 6.76667

3 Std. Dev. of y p 2.60128

4 Margin of Error 8.27844

5 Lower Limit 16.72

6 Upper Limit 33.28

Prediction Interval for yp

The 95% prediction interval estimate of the number of cars sold in one particular week when 3 TV ads are run is:

Prediction Interval for yp

25 + 8.28 = 16.72 to 33.28 cars

Residual Analysis

ˆ i iy y

 Much of the residual analysis is based on an examination of graphical plots.

 Residual for Observation i

 The residuals provide the best information about e .

 If the assumptions about the error term e appear questionable, the hypothesis tests about the significance of the regression relationship and the interval estimation results may not be valid.

Residual Plot Against x

 If the assumption that the variance of e is the same for all values of x is valid, and the assumed regression model is an adequate representation of the relationship between the variables, then

The residual plot should give an overall

impression of a horizontal band of points

ˆy y

Good Pattern R

e si

d u

a l

Residual Plot Against x

ˆy y

R e si

d u

a l

Nonconstant Variance

Residual Plot Against x

ˆy y

R e si

d u

a l

Model Form Not Adequate

 Residuals

Observation Predicted Cars Sold Residuals

1 15 -1

2 25 -1

3 20 -2

4 15 2

5 25 2

Residual Plot Against x

TV Ads Residual Plot

-3

-2

-1

0 1 2 3 4 TV Ads

R e

s id

u a

Residual Analysis: Autocorrelation

 Often, the data used for regression studies in business and economics are collected over time.

 It is not uncommon for the value of Y at one time period to be related to the value of Y at previous time periods.

 In this case, we say autocorrelation (or serial correlation) is present in the data.

Residual Analysis: Autocorrelation

 With positive autocorrelation, we expect a positive residual in one period to be followed by a positive residual in the next period.

 With positive autocorrelation, we expect a negative residual in one period to be followed by a negative residual in the next period.

 With negative autocorrelation, we expect a positive residual in one period to be followed by a negative residual in the next period, then a positive residual, and so on.

Residual Analysis: Autocorrelation

 When autocorrelation is present, one of the regression assumptions is violated: the error terms are not independent.

 When autocorrelation is present, serious errors can be made in performing tests of significance based upon the assumed regression model.

 The Durbin-Watson statistic can be used to detect first-order autocorrelation.

 Durbin-Watson Test Statistic

Residual Analysis: Autocorrelation

e e

t t t

t t

n 





 



( )1 2

e e

t t t

t t

n 





 



( )1 2

  ˆi i ie y yThe ith residual is denoted

Residual Analysis: Autocorrelation

 Durbin-Watson Test Statistic

• A value of two indicates no autocorrelation.

• If successive values of the residuals are close together (positive autocorrelation is present), the statistic will be small.

• The statistic ranges in value from zero to four.

• If successive values are far apart (negative autocorrelation is present), the statistic will be large.

 Suppose the values of e (residuals) are not independent but are related in the following manner:

where r is a parameter with an absolute value less than one and zt is a normally and independently distributed random variable with a mean of zero and variance of  2.

 We see that if r = 0, the error terms are not related.

Residual Analysis: Autocorrelation

et = r et-1 + zt

 The Durbin-Watson test uses the residuals to determine whether r = 0.

Residual Analysis: Autocorrelation

 The null hypothesis always is:

 The alternative hypothesis is:

to test for positive autocorrelation a : 0H r 

to test for negative autocorrelation a : 0H r 

to test for pos. or neg. autocorrelation a : 0H r 

r 0 : 0H there is no autocorrelation

Residual Analysis: Autocorrelation

A Sample Of Critical Values For The Durbin-Watson Test For Autocorrelation

Significance Points of dL and dU:  = .05

Number of Independent Variables

1 2 3 4 5

n dL dU dL dU dL dU dU dU dU dU

15 1.08 1.36 0.95 1.54 0.82 1.75 0.69 1.97 0.56 2.21

16 1.10 1.37 0.98 1.54 0.86 1.73 0.74 1.93 0.62 2.15

17 1.13 1.38 1.02 1.54 0.90 1.71 0.78 1.90 0.67 2.10

18 1.16 1.39 1.05 1.53 0.93 1.69 0.82 1.87 0.71 2.06

Positive autocor- relation

Incon- clusive

No evidence of positive autocorrelation

Residual Analysis: Autocorrelation

0 dL dU 2 44-dL4-dU

Negative autocor- relation

Incon- clusive

No evidence of negative autocorrelation

0 dL dU 2 44-dL4-dU

Incon- clusive

No evidence of autocorrelation

0 dL dU 2 44-dL4-dU

Incon- clusive

Negative autocor- relation

Positive autocor- relation

End of Chapter 14