Project Part 4
Myhome2020@
regressionassumptions.pptAssumptions of the Linear Regression Model
- I. Assumptions of the Linear Regression Model
- A. Overview
- Develop techniques to test hypotheses about the parameters of our regression line.
- For instance, one important question is whether the slope of the regression line is statistically distinguishable from zero?
- As we showed last time, if the independent variable has no effect on the dependent variable then b should be close to zero.
Assumptions of the Linear Regression Model
- A. Overview (cont.)
- So testing if beta=0 is really a test to see if the independent variable has any effect on the dependent variable.
Assumptions of the Linear Regression Model
- B. OLS Line
- 1. The predicted line
- The standard regression line that we had seen from last time is just
- Think of this regression line as the expected value of Y for a given value of X.
Assumptions of the Linear Regression Model
- 2. Error term
- To include this in our analysis we use the equation
- where ei is the error in observation i.
- These epsilons (errors) are random variables and each have there own distribution.
Assumptions of the Linear Regression Model
- C. Assumptions
To be explicit, the analysis that follows makes the following four assumptions.
- 1. Linearity
- The true relation between Y and X is captured in the equation: Y = a + bX
- 2. Homoskedasticity (Homogeneous Variance)
- Each of the ei has the same variance.
Assumptions of the Linear Regression Model
- 3. Independence
- Each of the ei's is independent from each other. That is, the value of one does not effect the value of any other observation i's error.
- 4. Normality
- Each ei is normally distributed
Assumptions of the Linear Regression Model
- Combine assumption 4 with assumption 2, this means that the error terms are normally distributed with mean = 0 and variance s2
Assumptions of the Linear Regression Model
- Notice that s2, the variance of the e's, is the expected squared deviation of the Y's from the regression line.
Assumptions of the Linear Regression Model
- D. Examples of Violations
- 1. Linearity
- The true relation between the independent and dependent variables may not be linear.
- For example, Consider campaign fundraising and the probability of getting of winning an election.
Assumptions of the Linear Regression Model
- 2. Homoskedasticity
- This assumption means that we do not expect to get larger errors in some cases than in others.
- Of course, due to the luck of the draw, some errors will turn out to be larger then others. But homoskedasticity is violated only when this happens in a predictable manner.
Assumptions of the Linear Regression Model
- 2. Homoskedasticity (cont.)
- Example: income and spending on certain goods.
- People with higher incomes have more choices about what to buy.
- We would expect that there consumption of certain goods is more variable than for families with lower incomes.
Assumptions of the Linear Regression Model
Assumptions of the Linear Regression Model
- 3. Independence
- The independence assumption means that two variables will not necessarily influence one another.
- The most common violation of this occurs with data that are collected over time or time series analysis.
- Example: high tariff rates in one period are often associated with very high tariff rates in the next period.
- Example: Nominal GNP
- 4. Normality
- Of all the assumptions, this is the one that we need to be least worried about violating. Why?
Assumptions of the Linear Regression Model
- 2. Recap
There are two things that we must be clear on:
- First, we never observe the true population regression line or the actual errors.
- Second, the only information that we know are Y observations and the resulting fitted regression line.
- We observe only X and Y from which we estimate the regression and the distribution of the error terms around the observed data.
Discussion of Error term
- II. Discussion of Error term
- A. Regression as an approximation
- 1. The Fitted Line
- The least squares method is the line that fits that data with the minimum amount of variation.
- That is, the line that minimizes the sum of squared deviations.
- a. Remember the criterion:
Discussion of Error term
- 1. The Fitted Line (cont.)
- b. Approximation
- But the regression equation is just an approximation, unless all of the observed data fall on the predicted line
Discussion of Error term
- 2. The Fitted line with variance
- a. Analogy to point estimates
- With point estimates, we did not expect that all the observed data would fall exactly on the mean.
- Rather, the mean represented the expected or the average value.
- Similarly, then, we do not expect that the regression line be a perfect estimate of the true regression line. There will be some error.
Discussion of Error term
- 2. The Fitted line with variance (cont.)
Discussion of Error term
- b. Measurement of variance
- We need to developed a notion of dispersion or variance around the regression line.
- A measurement of the error of the regression line is
Discussion of Error term
- B. Causes of Error
This error term is the result of two components:
- 1. Measurement Error
- Error that results from collecting data on the dependent variable.
- For example, there may be difficulty in measuring crop yields due to sloppy harvesting.
- Or perhaps two scales are calibrated slightly differently.
Discussion of Error term
- 2. Inherent Variability
- Variations due to other factors beyond the experimenter's control.
- What would be some examples in our fertilizer problem?
Rain
Land conditions
Discussion of Error term
- C. Note:
- The error term is very important for regression analysis.
- Since the rest of the regression equation is completely deterministic, all unexpected variation is included in the error term.
- It is this variation that provides the basis of all estimation and hypothesis testing in regression analysis.
Fertilizer
Yield
a
intercept
b=0
Y=a+bX
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
$
Y
a
bX
=
+
b=
xy
x
å
å
2
a
Y
b
X
=
-
Y
X
i
i
i
=
+
+
a
b
e
Fertilizer
Yield
a
intercept
b=0
Y=a+bX
X
X
X
X
X
X
X
X
X
X
X
X
X1
X3
X
X
X
X
X
X
X
e
2
e
2
e
3
2
E(
e
i
2
)=
s
2
for all i
E(
e
i
*
e
j
) = 0 for i
¹
j
m
X
m=0
s
2
We write this as
e
i
~ N(0,
s
2
)
s
e
2
2
2
2
=
-
=
-
+
=
E
Y
Y
E
Y
a
bX
E
i
i
i
i
i
(
$
)
(
(
))
(
)
.
$50,000
P(w)
Spending
X
X
X
X
X
X
X
X
X
1
2
3
4
5
6
X
9
8
7
10
e
9
e
8
e
7
e
5
e
6
income
Spending
(
)
Minimize
Y
Y
d
S
S
2
2
=
-
$
Yield
Fertilizer
40
50
60
70
80
100
200
300
400
500
600
700
x
x
x
x
x
x
x
a
change in x
change in Y
Y=a+bX
Yield
Fertilizer
40
50
60
70
80
100
200
300
400
500
600
700
x
x
x
x
x
x
x
a
change in x
change in Y
ei
e
Y
Y
i
i
i
=
-
$
