Statistic

giumag

Lecture_Slides_Ch_13_MTH_305_Statistics.ppt

Home >Mathematics homework help >Statistics homework help >Statistic

Chap 13-*

Chapter 13

Regression Analysis:

PART 1: Simple Linear Regression

Basic Business Statistics
10th Edition

Chap 13-*

Learning Objectives

In this chapter, you learn:

How to use regression analysis to predict the value of a dependent variable (Y) based on an independent variable (X): X causes Y
How to evaluate the assumptions of regression analysis and know what to do if the assumptions are violated
To make inferences about the slope in a linear regression (linear relation b/w X and Y)
To estimate mean values and predict individual values

Chap 13-*

Analysis when Two Variables are Related

A scatter diagram can be used to show the relationship between two variables

Scatter diagrams were first presented in Ch. 2

Correlation analysis is used to measure strength of the association (linear relationship) between two variables (Ch. 3)

Correlation is only concerned with strength of the relationship
No causal effect is implied with correlation

Regression analysis is used to show causation

Changes in X cause changes in Y

Correlation Coefficient (r)

–1 < r < 1
The closer to –1, the stronger the negative linear relationship
The closer to 1, the stronger the positive linear relationship
The closer to 0, the weaker the linear relationship

Chap 13-*

Chap 3-*

Scatter Plots of Data with Various Correlation Coefficients

r = -1

r = -.6

r = 0

r = +.3

r = +1

r = 0

Chap 13-*

Introduction to
Regression Analysis

Regression analysis is used to:
Explain the impact of changes in an independent variable on changes in the dependent variable
Predict the value of a dependent variable based on the value of one or more independent variables

Dependent variable (Y): the variable we wish to predict or explain

Independent variable (X): the variable used to explain the dependent variable

Chap 13-*

Simple Linear Regression Example

A real estate agent wishes to examine the relationship between the selling price of a home and its size (measured in square feet)

Dependent variable (Y) = house price
Independent variable (X) = square feet

Chap 13-*

Sample Data for House Price Model

House Price (Y)	Square Feet (X)
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Chap 13-*

Types of Relationships

Linear relationships

Non-linear relationships

Chap 13-*

Simple Linear Regression Model

Only one independent variable, X
Relationship between X and Y is described by a linear function:

Y = intercept + slope(X)

Y = a + bX

Changes in Y are assumed to be caused by changes in X

Chap 13-*

Linear Relationships

Strong relationships

Weak relationships

Chap 13-*

Linear Relationships

No relationship

(continued)

Chap 13-*

Linear component

Simple Linear Regression Model

Population
Y intercept

Population Slope
Coefficient

Random Error term

Dependent Variable

Independent Variable

Random Error

component

Population Parameters

Chap 13-*

The simple linear regression equation provides an estimate of the population regression line

Sample Statistics:
Regression Equation

Estimate of the regression
intercept

Estimate of the regression slope

Estimated (or predicted) Y value for observation i

Value of X for observation i

Formulas: Slope and Intercept

Slope:

b1 =

Intercept:

b0 =

Chap 13-*

INTERCEPT: b0 is the estimated average value of Y when the value of X is zero

SLOPE: b1 is the estimated change in the average value of Y as a result of a one-unit change in X

Interpretation of the
Slope and the Intercept

Chap 13-*

Simple Linear Regression Example

A real estate agent wishes to examine the relationship between the selling price of a home and its size (measured in square feet)

A random sample of 10 houses is selected
Dependent variable (Y) = house price in $1000s
Independent variable (X) = square feet

Chap 13-*

Sample Data for House Price Model

House Price in $1000s (Y)	Square Feet (X)
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Chap 13-*

Graphical Presentation

House price model: scatter plot

Chart2

1400
1600
1700
1875
1100
1550
2350
2450
1425
1700

House Price

Square Feet

House Price ($1000s)

245

312

279

308

199

219

405

324

319

255

Sheet4

SUMMARY OUTPUT
Regression Statistics
Multiple R	0.76211
R Square	0.58082
Adjusted R Square	0.52842
Standard Error	41.33032
Observations	10
ANOVA
	df	SS	MS	F	Significance F
Regression	1	18934.9348	18934.9348	11.08476	0.01039
Residual	8	13665.5652	1708.1957
Total	9	32600.5000
	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580
RESIDUAL OUTPUT
Observation	Predicted House Price	Residuals
1	251.9231625835	-6.9231625835
2	273.8767101495	38.1232898505
3	284.8534839325	-5.8534839325
4	304.0628380528	3.9371619472
5	218.9928412345	-19.9928412345
6	268.388323258	-49.388323258
7	356.2025135221	48.7974864779
8	367.1792873051	-43.1792873051
9	254.6673560293	64.3326439707
10	284.8534839325	-29.8534839325

Sheet4

1400	1400
1600	1600
1700	1700
1875	1875
1100	1100
1550	1550
2350	2350
2450	2450
1425	1425
1700	1700

House Price

Predicted House Price

Square Feet

House Price

Square Feet Line Fit Plot

245

312

279

308

199

219

405

324

319

255

Sheet1

House Price	Square Feet
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Sheet1

0
0
0
0
0
0
0
0
0
0

House Price

Square Feet

House Price

Sheet2

Sheet3

Chap 13-*

Regression Using Excel

Data / Data Analysis / Regression

Chap 13-*

Excel Output

The regression equation is:

Regression Statistics
Multiple R	0.76211
R Square	0.58082
Adjusted R Square	0.52842
Standard Error	41.33032
Observations	10
ANOVA	df	SS	MS	F	Significance F
Regression	1	18934.9348	18934.9348	11.0848	0.01039
Residual	8	13665.5652	1708.1957
Total	9	32600.5000
	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580

Chap 13-*

Graphical Presentation

House price model: scatter plot and regression line

Slope

= 0.10977

Intercept

= 98.248

Chart2

1400
1600
1700
1875
1100
1550
2350
2450
1425
1700

House Price

Square Feet

House Price ($1000s)

245

312

279

308

199

219

405

324

319

255

Sheet4

SUMMARY OUTPUT
Regression Statistics
Multiple R	0.76211
R Square	0.58082
Adjusted R Square	0.52842
Standard Error	41.33032
Observations	10
ANOVA
	df	SS	MS	F	Significance F
Regression	1	18934.9348	18934.9348	11.08476	0.01039
Residual	8	13665.5652	1708.1957
Total	9	32600.5000
	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580
RESIDUAL OUTPUT
Observation	Predicted House Price	Residuals
1	251.9231625835	-6.9231625835
2	273.8767101495	38.1232898505
3	284.8534839325	-5.8534839325
4	304.0628380528	3.9371619472
5	218.9928412345	-19.9928412345
6	268.388323258	-49.388323258
7	356.2025135221	48.7974864779
8	367.1792873051	-43.1792873051
9	254.6673560293	64.3326439707
10	284.8534839325	-29.8534839325

Sheet4

1400	1400
1600	1600
1700	1700
1875	1875
1100	1100
1550	1550
2350	2350
2450	2450
1425	1425
1700	1700

House Price

Predicted House Price

Square Feet

House Price

Square Feet Line Fit Plot

245

312

279

308

199

219

405

324

319

255

Sheet1

House Price	Square Feet
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Sheet1

0
0
0
0
0
0
0
0
0
0

House Price

Square Feet

House Price

Sheet2

Sheet3

Chap 13-*

Interpretation of the
Intercept, b0

b0 is the estimated average value of Y when the value of X is zero (if X = 0 is in the range of observed X values)
Here, no houses had 0 square feet, so b0 = 98.24833 just indicates that, for houses within the range of sizes observed, $98,248.33 is the portion of the house price not explained by square feet

Chap 13-*

Interpretation of the
Slope Coefficient, b1

b1 measures the estimated change in the average value of Y as a result of a one-unit change in X
Here, b1 = .10977 tells us that the average value of a house increases by .10977($1000) = $109.77, on average, for each additional one square foot of size

Chap 13-*

Predict the price for a house with 2000 square feet:

The predicted price for a house with 2000 square feet is 317.85($1,000s) = $317,850

Predictions using
Regression Analysis

Chap 13-*

The coefficient of determination is the portion of the total variation in the dependent variable that is explained by variation in the independent variable
The coefficient of determination is also called R-squared and is denoted as r2 (also, R2)

Coefficient of Determination (r2)

note:

Chap 13-*

r2 = 1

Examples of Approximate
r2 Values

r2 = 1

Perfect linear relationship between X and Y:

100% of the variation in Y is explained by variation in X

Chap 13-*

Examples of Approximate
r2 Values

0 < r2 < 1

Weaker linear relationships between X and Y:

Some but not all of the variation in Y is explained by variation in X

Chap 13-*

Examples of Approximate
r2 Values

r2 = 0

No linear relationship between X and Y:

The value of Y does not depend on X. (None of the variation in Y is explained by variation in X)

r2 = 0

Chap 13-*

R-squared from Excel Output

58.08% of the variation in house prices is explained by variation in square feet

Regression Statistics
Multiple R	0.76211
R Square	0.58082
Adjusted R Square	0.52842
Standard Error	41.33032
Observations	10
ANOVA	df	SS	MS	F	Significance F
Regression	1	18934.9348	18934.9348	11.0848	0.01039
Residual	8	13665.5652	1708.1957
Total	9	32600.5000
	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580

Chap 13-*

Assumptions of Regression

Use the acronym LINE:

Linearity
The underlying relationship between X and Y is linear
Independence of Errors
Error values are statistically independent
Normality of Error
Error values (ε) are normally distributed for any given value of X
Equal Variance (Homoscedasticity)
The probability distribution of the errors has constant variance

Chap 13-*

Hypothesis testing about the Slope: t Test

t test for a population slope
Is there a linear relationship between X and Y?
Null and alternative hypotheses

H0: β1 = 0 (no linear relationship)

H1: β1  0 (linear relationship does exist)

Test statistic

where:

b1 = regression slope

coefficient

β1 = hypothesized slope

Sb = standard

error of the slope

Chap 13-*

Standard Error
of the Slope

The standard error of the regression slope coefficient (b1) is estimated by

where:

= Estimate of the standard error of the least squares slope

= Standard error of the estimate

Chap 13-*

Standard Error from
Excel Output

Regression Statistics
Multiple R	0.76211
R Square	0.58082
Adjusted R Square	0.52842
Standard Error	41.33032
Observations	10
ANOVA	df	SS	MS	F	Significance F
Regression	1	18934.9348	18934.9348	11.0848	0.01039
Residual	8	13665.5652	1708.1957
Total	9	32600.5000
	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580

Chap 13-*

Simple Linear Regression Equation:

The slope of this model is 0.1098

Does square footage of the house affect its sales price at 95% CL?

Hypothesis testing about the Slope: t Test

House Price (y)	Square Feet (x)
245	1400
312	1600
279	1700
308	1875
199	1100
219	1550
405	2350
324	2450
319	1425
255	1700

Chap 13-*

Example

H0: β1 = 0

H1: β1  0

From Excel output:

(continued)

	Coefficients	Standard Error	t Stat	P-value
Intercept	98.24833	58.03348	1.69296	0.12892
Square Feet	0.10977	0.03297	3.32938	0.01039

Critical Value for t statistic

Use “t” table on page 814-815

Depends on:

degrees of freedom: df = n – 2

Significance level: 𝛼

95% CL has 𝛼 = 0.05
90% CL has 𝛼 = 0.10

Chap 13-*

Chap 8-*

t Table (p. 814-815)

Upper Tail Area

.25

…

.025

1.000

…

12.7062

…

0.7064

…

2.3060

The body of the table contains t values, not probabilities

Let: n = 10
df = n - 2 = 8

95% CL
 = 0.05
/2 = 0.025

/2 = 0.025

Chap 13-*

Result

H0: β1 = 0

H1: β1  0

Test Statistic: t = 3.329

There is sufficient evidence that square footage affects house price

From Excel output:

Reject H0

Decision:

Conclusion:

Reject H0

a/2=.025

-tα/2

Do not reject H0

tα/2

a/2=.025

-2.3060

2.3060

3.329

d.f. = 10-2 = 8

(continued)

	Coefficients	Standard Error	t Stat	P-value
Intercept	98.24833	58.03348	1.69296	0.12892
Square Feet	0.10977	0.03297	3.32938	0.01039

Chap 13-*

Confidence Interval Estimate
for the Slope

Confidence Interval Estimate of the Slope:

Excel Printout for House Prices:

At 95% level of confidence, the confidence interval for the slope is (0.0337, 0.1858)

d.f. = n - 2

	Coefficients	Standard Error	t Stat	P-value	Lower 95%	Upper 95%
Intercept	98.24833	58.03348	1.69296	0.12892	-35.57720	232.07386
Square Feet	0.10977	0.03297	3.32938	0.01039	0.03374	0.18580

Confidence Interval of Slope

Since the confidence interval from above does not contain the 0, we can reject the null
CONCLUSION: We are 95% confident that there is a positive linear relationship between the size of a house and its price.

Chap 13-*

Pitfalls of Regression Analysis

Lacking an awareness of the assumptions underlying regression methodology
Not knowing how to evaluate the assumptions
Not knowing the alternatives to regression if a particular assumption is violated
Using a regression model without knowledge of the subject matter
Extrapolating outside the relevant range

Chap 13-*

Strategies for Avoiding
the Pitfalls of Regression

Start with a scatter diagram of X vs. Y to observe possible relationship
Perform residual analysis to check the assumptions
Plot the residuals vs. X to check for violations of assumptions such as homoscedasticity
Use a histogram, stem-and-leaf display, box-and-whisker plot, or normal probability plot of the residuals to uncover possible non-normality

Chap 13-*

Strategies for Avoiding
the Pitfalls of Regression

If there is violation of any assumption, use alternative methods or models
If there is no evidence of assumption violation, then test for the significance of the regression coefficients and construct confidence intervals
Avoid making predictions or forecasts outside the relevant range

(continued)

_______

__________

)

)(

(

)

(

100

150

200

250

300

350

400

450

050010001500200025003000

Square Feet

House Price ($1000s)

feet)

(square

0.10977

98.24833

price

house

100

150

200

250

300

350

400

450

050010001500200025003000

Square Feet

House Price ($1000s)

feet)

(square

0.10977

98.24833

price

house

feet)

(square

0.10977

98.24833

price

house

317.85

0.1098(200

98.25

(sq.ft.)

0.1098

98.25

price

house

squares

sum

total

squares

sum

regression

SST

SSR

0.58082

32600.5000

18934.9348

SST

SSR

d.f.

)

SSX

SSE

0.03297

(sq.ft.)

0.1098

98.25

price

house

32938

03297

10977