Forecasting and Business Analysis

Writer Pro

fbasourdintopic03_handouts.pdf

Home >Business & Finance homework help >Forecasting and Business Analysis

22/08/13

Forecas/ng and Business Analysis

Topic 3

This topic

•  Koop Ch 6 – Mul/ple regression

•  Last topic: – Simple regression

Mul/ple Regression

22/08/13

Differences between simple and mul6ple regression

•  Mul/ple regression is like simple regression, except that there are many explanatory variables: X1, X2,..., Xk

•  The key differences are: – You can perform mul/ple t-‐tests, achieve higher R-‐ squared, and build a more theore/cally complete model of Y

– The effect of each independent variable on Y is es/mated CONDITIONAL on the other independent variables

OLS es6ma6on

•  Mul/ple regression model:

•  OLS es/mates: •  These es/mates (s/ll) minimise the sum of squared residuals

•  Solu/on to minimisa/on problem: Messy •  Calcula/on of is harder for mul/ple OLS •  Excel will calculate the OLS es/mates for you

Yi = α + β1X1i +…+ βkXk + ei α̂, β̂1,…, β̂k

β̂

Sta6s6cal Aspects and Evalua6on

•  Standard error of the es/mate : largely the same as for simple regression, just with bigger ‘k’

•  Confidence intervals can be calculated for each individual coefficient, as we did before for just the one coefficient.

•  Can test βj=0 using a t-‐test for each individual coefficient (j=1,2,..,k), just as before

22/08/13

Mul6ple OLS sta6s6cs – cont’d

•  R2 is s/ll a measure of fit, with the same interpreta/on (although now it is no longer simply the square of the correla/on between Y and 'X').

•  Can s/ll test R2=0 using an F-‐test, but with bigger 'k'. •  If you find R2>0, then you conclude that the explanatory variables together provide explanatory power (note: this does not necessarily mean that each individual explanatory variable [through t-‐stats] is significant).

Interpre6ng OLS Es6mates in the Mul6ple Regression Model

Mathema/cal Intui/on Total vs. par/al deriva/ve Simple regression: Mul/ple regression:

dY dX

= β

∂Y ∂Xj

= β j

Interpreta6on of Mul6ple OLS Es6mates, cont’d

•  Verbal intui/on •  the marginal effect of Xj on Y, ceteris paribus

•  is the effect on the dependent variable of a small change in the jth explanatory variable, holding all the other explanatory variables constant.

β j

22/08/13

Example: Explaining Birth Weight

•  Let’s take some 6me going over the following example:

•  Data on N = 1388 individuals •  Dependent variable: – Y = birth weight of child, in pounds

•  Explanatory variables: – X1 = number of cigareges smoked per day by pregnant mum

– X2 = Family income, 1988$USD •  NOTE k=2!

Example: Excel Output

Explaining Birth Weight

•  Figed Regression Line:

•  Evaluate the following: – Significance of coefficient es/mates – R2 and its significance – Do the results accord with common sense and/or formal theory in the areas of economics, general human behaviour, and health?

Ŷ = 7.31− 0.029X1 + 0.000006X2

22/08/13

Interpreta6on: Birth Weight Results

•  Since , then at the average number of cigareges smoked: – Mum having one extra cigarege per day is expected to reduce the baby's birth weight by 0.029 pounds, ceteris paribus (i.e., holding income constant)

– If we compare individuals with the exact same income, mums who smoke 10 cigareges per day are expected to have babies that weigh 0.29 pounds less than those of mums who do not smoke.

β̂1 = −0.029

Interpreta6on: Birth Weight Results

•  Since , then at the average family income: – The family's having one extra dollar of annual income is expected to increase the baby's birth weight by 0.000006 pounds, ceteris paribus (i.e., holding mum‟s smoking behaviour constant)

–  If we compare mums with the exact same number of cigareges smoked per day, mums who have $10,000 more in family income are expected to have babies that weigh 0.06 pounds more (a seemingly “small” effect in the output, but economically meaningful and sta/s/cally significant!).

β̂2 = 0.000006

Example: Data transforma6on

•  Is it reasonable to expect that birth weight will increase at a constant rate with income? Economic intui/on would tell us that this is probably not the case.

•  Can check X-‐Y plot of birth weight and income to confirm.

•  Might make more sense to use ln(income)?

22/08/13

Excel Output

Interpreta6on of Results

•  No massive changes in the explanatory power of the model, and no changes to significance

•  New figed model is:

•  The es/mated coefficient on cigs (number of cigareges) has not changed all that much, nor has the intercept.

Ŷ = 7.31− 0.029X1 + 0.116X2

Interpre6ng

•  How do we interpret the new coefficient on our transformed variable?

•  Recall, , our dependent variable is in pounds, and our independent variable is ln(income)

•  Interpreta/on: A 1% increase in income is expected to increase birth weight by about (0.116/100) = .00116 pounds, ceteris paribus (i.e., holding smoking behaviour constant).

β̂2

β̂2 = 0.116

22/08/13

Some piJalls…?

•  Consider the following output for a simple version of our birth weight model:

Answer

•  These es/mators come from two different regressions which control for different explanatory variables.

•  This means that the two es/mates come with different ‘ceteris paribus’ condi/ons. – Specifically: In the simple model, we are not holding anything else constant.

Answer (cont’d) •  Simple Regression: 1% increase in income is expected to increase birth weight by 0.00147lbs

•  One solu/on to increasing birth weight, according to this model, would be to give people more money.

•  BUT: other factors will influence birth weight, such as smoking, diet, etc.

•  People with higher incomes tend not to be smokers and therefore are “healthier”

•  The simple model only shows that “richer” people tend to have higher birth weight babies – but gives no indica/on that this effect may be partly due to their increased health

22/08/13

Sta6s6cal Evidence

•  The nega/ve correla/on between income and cigarege consump/on suggests that people with lower incomes tend to consume more cigareges, and vice versa. We are not examining the smoking and income effects separately when we only consider one variable in our model.

Mul6ple regression may provide a bePer approxima6on to reality

•  If we evaluate the overall performance and fit of the model, we see that the mul/ple regression model has a lower standard error, higher F sta/s/cs, and a higher Adjusted R2. –  Adjusted R squared accounts for the fact that more variables included (greater k)

•  Models which contain all or most of the drivers of Y will tend to look beger “on paper,” as well as in terms of their theore/cal coherence, than simple regression.

•  This is not always the case, it depends on the marginal effect of each addi/onal variable (the trade-‐off between an increase in R2 and the increase in k)

PiJall #1: OmiPed Variables Bias The technical term for what we just described is “OMITTED VARIABLES BIAS” IF •  We exclude explanatory variable(s) that should be present in

the model

AND •  •These variable(s) are correlated with an included explanatory

variable

THEN •  The OLS es/mate of the coefficient on the included explanatory

variable will be “biased” – that is, it won’t reflect the “pure” impact of that variable on Y

22/08/13

OVB cont’d

•  In our simple regression, we only considered ln(income)

•  Many important determinants of birth weight were omiged, such as smoking.

•  This omiged variable was correlated with income, and therefore the es/mate from our simple regression was biased.

OmiPed variable bias

•  True model is: •  But we omit and es/mate:

•  If and are correlated, OLS is biased

y = α + β1X1 + β2X2 + e

y = α + β1X1 + v v = β2X2 + e

X2 X1

OmiPed variable bias

Bias (+) Bias (–) Bias (–) Bias (+)

corr(X1,X2) > 0 corr(X1,X2) < 0 β2 > 0 β2 < 0

The true model is: But we omit ability. Since ability and education are most likely correlated (+ve), our coefficient on education is likely biased upwards.

wage = β0 + β1educ + v

wage = α + β1educ + β2ability + e

v = β2ability + e( )

22/08/13

PiJall #2: Irrelevant variables •  If we include any irrelevant variables as independent variables in our model, it will not cause bias if the true coefficient of the extra variable is zero (irrelevant).

•  However, it will increase the variance of the es/mated coefficients, which will tend to decrease the magnitude of their t-‐scores.

•  Irrelevant variables also usually decreases the adjusted R squared.

•  Hence irrelevant variables reduce the precision of regressions.

PiJall #3: Mul6-‐collinearity •  Explanatory variables may be HIGHLY correlated è the

model has trouble differen/a/ng between their effects on Y.

•  High R2, large F-‐stat, but insignificant t stats for coefficient es/mates (or wrong sign).

•  ie. Model overall fits well, but cannot pin down marginal effects of individual variables

•  Diagnosing: Look at your correla/on matrix for high levels of correla/on between your explanatory variables. This will reveal the source and extent of the mulCcollinearity problem.

•  NOTE: High correla/on between your dependent variable and independent variable(s) is usually OK!

Remedies for mul6collinearity

1.Do nothing – If looking for overall predic/on and not individual effects

– If theory suggests variables should be included 2.Transform variables – If theore/cally jus/fied. Mul/collinearity is a problem when there is a linear rela/onship between explanatory variables

3.Drop or combine explanatory variables

22/08/13

How do you select explanatory variables?

•  Include (insofar as possible) all explanatory variables that you think might explain your dependent variable. This will reduce OVB.

•  However, including irrelevant variables or ones that are highly mul/collinear is also not advisable.

•  Ideally, turn to theory, intui/on, logic, and/or common sense for sugges/ons on what is appropriate to include.

Example: Forecas6ng Demand •  Back to first year microeconomics. •  Theory argues that market demand for a product is a

func/on of the following: –  Price –  Tastes and preferences –  Disposable income –  Number of consumers in the market –  Prices of related goods –  Expecta/ons

•  This theory gives you an indica/on of what should be included in a regression model of market demand. However, you will also need to consider the availability of data (which is almost always the biggest constraint on model development).