Collection and Use of Data in Economics

Regression

Analysis

Population Linear Regression Model  X and Y relationship is described by a linear function.

 Changes in Y are assumed to be influenced by changes in X.

 Example: housing prices (y) and square feet (x)

House Price

in $1000s

(Y)

Square

Feet

(X)

245 1400

312 1600

279 1700

308 1875

199 1100

219 1550

405 2350

324 2450

319 1425

255 1700

0

50

100

150

200

250

300

350

400

450

0 500 1000 1500 2000 2500 3000

Square Feet

H o

u s e P

ri c e (

$ 1 0 0 0 s )

feet) (square 0.10977 98.24833 price house 

Slope

=0.10977

Intercept

=98.248

Sample Linear Regression Model

Total variation is made up of two parts:

SSE SSR SST  Sum of Squares

Total

Sum of Squares

Regression

Sum of Squares

Error (residual)

SST = (y i - y)

2å   2

ii )y(ySSE ˆ 

2

i )yy(SSR ˆ

Sample Linear Regression Model

 SST: Variation of yi values around

their mean, y.

 SSR: Explained variation due to the

linear relationship between x and y.

 SSE: Unexplained variation due to

factors other than the linear

relationship between x and y.

Coefficient of Determination (R2)

 The portion of the total variation in the dependent

variable that is explained by variation in the

independent variable.

1R0 2

squares of sum total squares of sum regression

SST

SSR R

2 

Excel Output

Regression Statistics

Multiple R 0.76211

R Square 0.58082

Adjusted R Square 0.52842

Standard Error 41.33032

Observations 10

ANOVA df SS MS F Significance F

Regression 1 18934.9348 18934.9348 11.0848 0.01039

Residual 8 13665.5652 1708.1957

Total 9 32600.5000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386

Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

58.08% of the variation in

house prices is explained by

variation in square feet

0.58082 32600.5000

18934.9348

SST

SSR R

2 

Standard Error

Y Y

X X e

s small e

s large

se is a measure of the variation of observed y values from

the regression line

The magnitude of se should always be judged relative to the size

of the y values in the sample data

i.e., se = $41.33K is large relative to house prices in the $200 - $300K range

Excel Output

Regression Statistics

Multiple R 0.76211

R Square 0.58082

Adjusted R Square 0.52842

Standard Error 41.33032

Observations 10

ANOVA df SS MS F Significance F

Regression 1 18934.9348 18934.9348 11.0848 0.01039

Residual 8 13665.5652 1708.1957

Total 9 32600.5000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386

Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

41.33032s e



Standard Error

Y

X

Y

X 1b

S small 1b

S large

is a measure of the variation in the slope of regression

lines from different possible samples 1b

S

Regression Statistics

Multiple R 0.76211

R Square 0.58082

Adjusted R Square 0.52842

Standard Error 41.33032

Observations 10

ANOVA df SS MS F Significance F

Regression 1 18934.9348 18934.9348 11.0848 0.01039

Residual 8 13665.5652 1708.1957

Total 9 32600.5000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386

Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

0.03297s 1b



Excel Output

 t test for a population slope  Is there a linear relationship between X and Y?

 Null and alternative hypotheses H0: β1 = 0 (no linear relationship)

H1: β1  0 (linear relationship does exist)

 Test statistic

Inference about the Slope: t Test

1b

11

s

βb t

 

2nd.f. 

where:

b1 = regression slope coefficient

β1 = hypothesized slope

sb1 = standard error of the slope

H0: β1 = 0

H1: β1  0

Test Statistic: t = 3.329

From Excel output:

Coefficients Standard Error t Stat P-value

Intercept 98.24833 58.03348 1.69296 0.12892

Square Feet 0.10977 0.03297 3.32938 0.01039

1b s t b1

Decision: Reject H0

Conclusion: There is sufficient

evidence that square footage affects a

homes price.

Reject H0 Reject H0

a/2=.025

-tn-2,α/2

Do not reject H0

0

a/2=.025

-2.3060 2.3060 3.329

d.f. = 10-2 = 8

t8,.025 = 2.3060

tn-2,α/2

Inference about the Slope: t Test

H0: β1 = 0

H1: β1  0

P-value = 0.01039

From Excel output:

Coefficients Standard Error t Stat P-value

Intercept 98.24833 58.03348 1.69296 0.12892

Square Feet 0.10977 0.03297 3.32938 0.01039

P-value

Decision: P-value < α so Reject H0

Conclusion: There is sufficient evidence

that square footage affects house price

Inference about the Slope: P value

Confidence Interval Estimate

11 bα/22,n11bα/22,n1 stbβstb

 

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386

Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580

d.f. = n - 2

This 95% confidence interval does not include 0.

Conclusion: There is a significant relationship between house price and

square feet at the .05 level.

we are 95% confident that the average impact on sales price

is between $33.70 and $185.80 per square foot of house size

Forecasting with Regression Analysis

The predicted price for a house with 2000 square feet is

98.24833+2000(.10977)=317,788

Multiple Regression

εXβXβXββY KK22110

 

Population Multiple Regression Equation with K Independent Variables:

Y-intercept Population slopes Random Error

KiK2i21i10i xbxbxbby  ˆ

Estimated (or predicted) value of y

Estimated slope coefficients

Sample Multiple Regression Equation with K Independent Variables:

Estimated intercept

Example: Julian Pie Company

Week

Pie

Sales

Price

($)

Advertising

($100s)

1 350 5.50 3.3

2 460 7.50 3.3

3 350 8.00 3.0

4 430 8.00 4.5

5 350 6.80 3.0

6 380 7.50 4.0

7 430 4.50 3.0

8 470 6.40 3.7

9 450 7.00 3.5

10 490 5.00 4.0

11 340 7.20 3.5

12 300 7.90 3.2

13 440 5.90 4.0

14 450 5.00 3.5

15 300 7.00 2.7

Sales = b0 + b1(Price) + b2(Advertising)

Excel Output

Regression Statistics

Multiple R 0.72213

R Square 0.52148

Adjusted R Square 0.44172

Standard Error 47.46341

Observations 15

ANOVA df SS MS F Significance F Regression 2 29460.027 14730.013 6.53861 0.01201

Residual 12 27033.306 2252.776

Total 14 56493.333

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 306.52619 114.25389 2.68285 0.01993 57.58835 555.46404

Price -24.97509 10.83213 -2.30565 0.03979 -48.57626 -1.37392

Advertising 74.13096 25.96732 2.85478 0.01449 17.55303 130.70888

44.2% of the variation in pie sales is explained by the variation in price

and advertising, taking into account the sample size and number of

independent variables

Adjusted Coefficient of Determination,

 R2 never decreases when a new X variable is added to the model, even if the new variable is not an important predictor variable

 This can be a disadvantage when comparing models

 What is the net effect of adding a new variable?

 We lose a degree of freedom when a new X variable is added

 Did the new X variable add enough explanatory power to offset the loss of one degree of freedom?

2 R