Forecasting and Business Analysis

Writer Pro

fbasourdintopic02_handouts.pdf

Home >Business & Finance homework help >Forecasting and Business Analysis

7/08/13

Topic 2

Simple Regression Koop Chapters 4 and 5

Admin

•  Assignment 1 to be posted week 3

•  Due date 13 September – week 7

•  Assign 1: Regression exercise and report

Last lecture

•  Maths and stats review – Data handling – Data descripKon

•  XY plots •  Mean, Standard deviaKion

– CorrelaKon – Probability and probability distribuKons

7/08/13

This topic

•  A discussion of the simple regression…

•  ABSOLUTELY ESSENTIAL READING: Koop Chapters 4 and 5

IntroducKon to Simple Regression

•  Regression is the most common tool of the applied economist.

•  Used to help understand what factors (variables) accountable for the outcome of variable of interest.

•  We begin with simple regression to understand the relaKonship between two variables, X and Y.

Imagine a “best-‐fiang” line… •  XY-‐plot of populaKon density against deforestaKon

7/08/13

The Regression Line IS the Line of Best Fit

•  The process of (bivariate) regression is the process of fiang a line through the points in the XY-‐plot that best captures the relaKonship between deforestaKon and populaKon density.

•  What do we mean by “best fiang” line?

Assumed Model Structure

Assume a true linear relaKonship exists between Y and X: Example: Y= output of a good, X= labour input α= ? (perhaps 0) β=? (perhaps 0.8= marginal product of labour)

Y = α + βX α = intercept of line β =slope of the line

NOT the Line of Perfect Fit 1.  Even if the straight line relaKonship were true

on average, we would never get all points on an XY-‐plot lying precisely on it due to the fact that some of Y’s movement is not able to be explained using X.

2.  Also – the true relaKonship is probably more complicated; a straight line is typically thought of as an approximaKon

3.  Y or X may be measured with errors. Due to 1, 2 and 3, we add an error term to the model.

7/08/13

Adding the error

Y =α + βX + e where e is an error.

•  What we know: X and Y. •  What we do not know: α, β and e. •  Regression analysis uses data (X and Y) to make an esKmate, of what α and β are.

•  NotaKon: and are the esKmates of α and β that the regression (line-‐fiang) process spits out.

α̂ β̂

Pre-‐ versus Post-‐EsKmaKon Model

•  True regression model:

•  EsKmated regression model:

Y = α + βX + e e = Y −α − βX e = error

Y = α̂ + β̂X + u

u = Y −α̂ − β̂X u = residual

How do we choose and ?

With more than two points, it’s usually not possible to find a line that fits perfectly through all points:

α̂ β̂

7/08/13

EssenKal CharacterisKc of Regression

(or “Ordinary Least Squares”)

OLS regression chooses the line that minimizes the sum of squared residuals.

Expressing the OLS esKmator

We observe data on two variables for i=1,..,N individuals. Each individual has a Yi and an Xi. Any line we fit/choice of and will yield residuals ui. OLS esKmator chooses and to minimise SSR

α̂ β̂

Sum of squared residuals = SSR = ui 2∑

α̂ β̂

MathemaKcal expressions for the bivariate OLS esKmators

SoluKon: and

β̂ = Yi −Y( ) Xi − X( )∑

Xi − X( ) 2∑

α̂ = Y − β̂X

7/08/13

Regression and CausaKon

How do you choose which variables to use? •  Ideally, the explanatory variable should be the one which causes/influences the other (dependent) variable: so, X causes Y.

•  If you can, only esKmate models where this causality assumpKon make sense.

•  But, what guides this? IntuiKon, reasoning raKonale, theory

Examples

•  Increases in X (= populaKon density) cause Y (= deforestaKon) to increase (or vice versa? Make your argument)

•  Increasing X (= the lot size of a house) causes Y (= its value) to increase (or vice versa? Make your argument)

•  Increasing X (= adverKsing expenditures) causes Y (= company sales) to increase (or vice versa? Make your argument)

Causality (cont.)

•  In pracKce, great care must be taken in interpreKng regression results as reflecKng causality. Why? –your assumpAon that X causes Y may be wrong. –X and Y may both be caused by some third factor, call it Z. –X may cause Y but Y may also cause X (e.g. exchange rates and interest rates). –the whole concept of causality may be inappropriate.

•  Formally, one key quesKon regression addresses is: “How much of the variability in Y can be explained by X?” (we will look at this shortly)

7/08/13

InterpretaKon of

•  EsKmated value of Y if X = 0 •  This is ooen not of interest Example: •  X = lot size, Y = house price •  = esKmated value of a house with lot size = 0

α̂

InterpretaKon of 1.  is the esKmate of the marginal effect of X on Y 2.  Using the regression model:

3.  The OLS esKmator – the esKmated “slope” – is a measure of how much Y tends to change when you change X.

4.  “If X changes by 1 unit then Y tends to change by units”, where “units” refers to what the variables are measured in (e.g. $, $billions, £, %, hectares, metres, etc.)

β̂

β̂ = dY dX

= ΔY ΔX

β̂

DeforestaKon example Development economists have theories that imply that increasing populaKon density should increase deforestaKon. Thus: •  Y = deforestaKon (annual percentage lost) = dependent variable

•  X = populaKon density (people per thousand hectares) = explanatory variable

•  Using data on N = 70 tropical countries we find:

= 0.000842 β̂

7/08/13

InterpretaKon and predicKon

a) “If populaKon density increases by 1 person per 1,000 hectares, then the average deforestaKon is esKmated (or expected) to increase by 0.000842 % per year” b) “If populaKon density increases by 100 people per 1,000 hectares, then deforestaKon is esKmated to increase by 0.0842% per year on average”

Basic evaluaKon staKsKcs

•  R-‐squared •  F-‐test •  Data evaluaKon •  t-‐test

R2: A Measure of Fit

IntuiKon: •  “Variability” = (e.g.) how deforestaKon rates vary across countries

Total variability in dependent variable Y = (1)+(2): 1.  Variability explained by the explanatory variable

(X) in the regression +

2. Variability that cannot be explained and is leo over in the residual.

7/08/13

Sums of squares

In mathemaKcal terms, TSS = RSS + SSR

where TSS = Total sum of squares = Note similarity to formula for variance.

TSS = Yi −Y( ) 2∑

More sums of squares

•  RSS = Regression sum of squares

•  SSR= Sum of squared residuals

RSS = Ŷi −Y( )∑ 2

SSR = u2 i=1

∑

R-‐squared expressed as sums of squares

•  R-‐squared is a measure of fit (i.e. how well does the regression line fit the data points – meaning how closely X and Y are related)

R2 = 1− SSR TSS

or equivalently: R2 = RSS TSS

since 1− SSR = RSS

7/08/13

ProperKes of R-‐squared

•  R2=1 means perfect fit. All data points exactly on the regression line (i.e. SSR=0).

•  R2= 0 means X does not have any explanatory power for Y whatsoever (i.e., X has no influence on Y).

•  Bigger values of R2 imply X has more explanatory power for Y.

•  R2 is equal to (the correlaKon between X and Y) squared (i.e. R2=r2xy)

0 ≤ R2 ≤1

R-‐squared example

•  R2 measures the proporKon of the variability in Y that can be explained by X.

Example: •  In regression of Y = deforestaKon on X = populaKon density, we obtain R2=0.44

àWe can say that “44% of the cross-‐country variaKon in deforestaKon rates can be explained by the cross-‐country variaKon in populaKon density”

F test of overall significance The F test is oHen used to measure the explanatory power of the whole model (or, equivalently, the significance of the R-‐ squared). The typical hypotheses in this context are: •  H0 : there is no staKsKcal significance on the relaKonship

between Y and X •  H1 : there is a staKsKcal significance on relaKonship between

Y and X •  The F staKsKc is calculated as the raKo of the amount of

variaKon in Y that is explained by the model to the amount of variaKon unexplained, corrected for degrees of freedom:

Fk,n−k−1 = RSS k

SSR n − k −1( )

7/08/13

DeforestaKon Excel output

Non lineariKes

August 13 32

average hourly earnings

0 5 10 15 20 years of education

August 13 33

100

200

300

child mortality

0 1000 2000 3000 4000 per capita gnp in 1980

7/08/13

August 13 34

EsKmates only!

As menKoned, and are es#mates of the true populaKon parameters only •  But how accurate are they? •  The t-‐test allows us to formally address this problem for each variable separately. It is based on the esKmated standard deviaKon – or “standard error” – of which is esKmated, along with the value itself, by the regression process

α̂ β̂

β̂

Standard error of

The s.e. of the esKmated slope varies: – directly with SSR (the variability in the residuals) – Inversely with N – Inversely with , which relates to the variance/variability of X

β̂

se = SSR

n − 2( ) X − X( )2∑

X − X( )2∑

7/08/13

What factors affect the accuracy of the esKmate ?

Ceteris paribus, •  A large number of observaKons (more data points)

•  Small errors (small SSR)

•  A bigger spread of values of the explanatory variable X (X has a range of values) will increase the accuracy of the esKmate

β̂

Very small sample size

Large sample size, large error variance

7/08/13

Large sample size, small error variance

Limited range of X values

DeforestaKon excel output

7/08/13

Test of a slope coefficient The t-‐test in the context of linear regression tests whether there is a staKsKcally significant linear relaKonship between X and Y. Hypotheses: To perform the test, form the following staKsKc using the esKmated coefficient and standrd error:

n-‐k-‐1 represents the degrees of freedom associated with the test; when there is one independent variable, k = 1.

H0 : β = 0 (No linear relationship) H1 : β ≠ 0 (Linear relationship)

t n−k−1( ) = β̂

se β̂( )

Concept check

Test your understanding of the t-‐test by doing the following: 1.  Calculate the t-‐stat using the standard error

of the esKmate and esKmated coefficient 2.  Check that this is equal to the t-‐stat reported

in excel

EvaluaKng the model using the original data: The issues

•  How well does the model ‘explain’ the variance in the dependent variable? –‘Goodness of fit’: the closer the points to the regression line, the bezer

•  How strongly are the independent variables related to the dependent variable? –Are the esKmated effects economically meaningful? –Are the esKmated effects staKsKcally significant?

•  Determine whether the underlying assumpKons of regression modelling have been met (more on this later…)

•  Determine robustness of the model to outliers (‘unusual’ observaKons)

7/08/13

Confidence in our results

•  Uncertainty about accuracy of the esKmator can be summarised in a “confidence interval”.

•  This will provide us with some more informaKon about the accuracy of our results

Confidence Interval for

•  Confidence interval for is given by:

Where: “criKcal value” from the t-‐distribuKon (note that Excel can provide the value) And is the standard error of

β̂ − t β̂ × se

β̂( ), β̂ + tβ̂ × seβ̂( )⎡⎣ ⎤⎦ β

t β̂ =

se β̂ = β̂

ConstrucKng a Confidence Interval •  If we want a 95% CI on , we need the s.e. (provided in excel

output) and the relevant t-‐staKsKc* •  Therefore, the CI on our esKmate is: Lower bound:

0.000842 – 1.99*0.0001165 = 0.00061

Upper bound: 0.000842 + 1.99*0.0001165 = 0.001075 CI = [0.00061, 0.001075]

InterpretaKon (informal):

–  There is a 95% probability that the true value of β lies between 0.00061 to 0.001075.

7/08/13

•  OLS esKmator has many nice staKsKcal properKes if certain condiKons hold.

•  These condiKons known as Gauss Markov CondiTons

Y = α + βX + e

The Gauss-‐Markov Assump#ons, In Brief

•  These necessary condiKons are: – The linear model is correct – We've got a random sample of data from the populaKon whose behaviour we're using the model to explain

– There's some sample variance in X – X and the unexplained part of Y (that is, e) aren't related

Why the G-‐M assump#ons ma<er

•  If any of these condiKons DON’T hold, then you can’t run the regression and expect the OLS esKmator to deliver parameter esKmates that are reasonable guesses of the true relaKonship between Y and X in the populaKon you care about. – Bias means is not a true esKmate of – Lack of precision: means has a large standard error relaKve to itself

β̂ β β̂

7/08/13