elem econ homework
shazuanzhe
Asymptoticpropertiesandheteroskedasticity.pptxAsymptotic properties and heteroskedasticity
Small vs. large sample properties of OLS
So far we have covered “small sample” properties of OLS (e.g. unbiasedness). We call these properties “small sample” properties because they apply regardless of the sample size. This is suggestive of the possibility of “large sample” properties that may differ from those we have discussed so far.
In fact, there are some nice large sample properties of OLS. As the sample size grows we can relax some of our CLM assumptions and still get some use out of OLS.
In particular, even if we do not necessarily have an unbiased estimator we may still be able to get “close” to the true parameter. Further, we can relax the very strong assumption of normally distributed errors and still conduct hypothesis testing.
We won’t get too technical with it (you’re welcome), but large sample properties (often called asymptotic properties) have something to do with limits. Specifically, we are interested in seeing what happens when the sample size goes to infinity.
CLM assumptions
Recall that the CLM assumptions consist of the 5 Gauss-Markov assumptions that ensure that OLS is BLUE.
Linearity: The true population model is given by:
Zero conditional mean:
No perfect collinearity
is a purely random sample from the population
Homoskedasticity:
Plus the assumption of normally distributed errors that, combined with the Gauss-Markov assumptions, ensures that OLS is BUE.
The population error e is independent of all of the regressors and
Pitfalls of OLS: biasedness
While unbiasedness is a lovely property, it is often very difficult to achieve in practice. In fact, some of the other estimators that are commonly used in econometrics to get around some pitfalls of OLS (like instrumental variables) are knowingly biased. So what gives; are econometricians just crazy?
Well maybe, but not in regards to this. In lieu of having unbiasedness, we can sometimes appeal to an asymptotic property called consistency.
Consistency
Here’s the idea underlying consistency: Let be an OLS estimator of for some j. Remember that is a random variable (a function of the random error term) with a probability distribution. If is consistent, then the probability distribution should collapse to a single point as the sample size, n, goes to infinity.
The formal definition of consistency involves a concept called probability limits, but I promised I wouldn’t get too technical. So it is enough to get the intuitive understanding of consistency.
Consistency of OLS
The conditions for consistency of the OLS estimator are similar to those necessary for unbiasedness, with one notable change.
Under the following conditions, the OLS estimator is a consistent estimator for
1. Linearity: The true population model is given by:
2.5 Zero mean and zero correlation: and for
3. No perfect collinearity
4. is a purely random sample from the population
Consistency of OLS
Notice that we replaced condition (2) of the Gauss-Markov assumptions, , with condition (2.5) for consistency.
Condition 2.5 is a weaker (less restrictive) assumption in the sense that the zero conditional mean assumption buys us the zero mean and correlation assumption. However, the reverse does not necessarily hold. If a model satisfies condition 2.5, that does not guarantee that it satisfies the zero conditional mean assumption.
The upshot is this: any estimator that is unbiased is also consistent, but we can have a consistent estimator that is actually biased.
Consistency: omitted variables
The key aspect of assumption 2.5 is that there must be zero correlation between all of the regressors and the error term. If the error term is correlated with any of the regressors then OLS is both biased and inconsistent.
The unfortunate implication is that large sample sizes do nothing for us when it comes to important omitted variables. If we fail to control for variables that are independently related to both the treatment variable and the control variable, then we will miss the mark with OLS.
Pitfalls of OLS: non-normal errors
In addition to the Gauss-Markov assumptions, we assumed normally distributed errors in order to conduct inference in OLS. So what happens if we do not have normally distributed errors? Can we still conduct hypothesis testing using standard normal, t, and F distribution as before?
In small samples, no we cannot. However, if we appeal to another large sample property, we can approximate the sampling distribution of OLS using those same hypothesis testing distributions in large samples regardless of what the actual sampling distribution is.
Asymptotic normality
The property we can appeal to is called asymptotic normality.
When we say that an estimator has an asymptotic distribution (such as a normal in this case) what this means is that as we increase the sample size to infinity, the actual distribution becomes closer and closer to the asymptotic distribution (and the limit is the asymptotic distribution).
We could describe the exact asymptotic distribution of the OLS estimator, but for our purposes all we are interested in is the following result. Under the Gauss-Markov assumptions, is asymptotically distributed .
Again, we won’t get technical, but this is a result of the Central Limit Theorem for the curious.
Asymptotic normali(t)y
We previously noted that a t distribution becomes arbitrarily close to a standard normal distribution as we jack up the sample size. Thus it is also valid to say that is asymptotically distributed .
Why is this important (other than because I say it is)? Because this allows us to use the t (or standard normal) distribution as an approximation for the true distribution. This also holds for F statistics. So as long as we can appeal to large sample properties, we can still conduct hypothesis tests exactly as we did before.
How big is big enough?
Of course, in order for the standard normal, t, or F distributions to be exactly appropriate we would have to have an infinite sample (after all these properties are results of limits). Good luck getting an infinite sample (spoiler alert: you can’t).
When we appeal to these large sample properties, we are doing so as an approximation. Then how large of a sample size do we need for it to be a “good enough” approximation to justify standard inference procedures?
That’s a fine question without an exact answer. Unfortunately, the error term is unobservable, so we have no idea what the exact distribution is (usually). Some distributions will become closer to the standard normal “quicker” (with smaller samples) than others.
Some econometricians have proposed a rules of thumb such as df=30 as a minimum, though this is far from fool-proof.
Ultimately, the larger your sample size, the better will be the approximation.
Pitfalls of OLS: heteroskewhatchmacallit
We return to the simple land of small samples to ask the question “what if we have heteroskedasticity?”.
For ease of exposition we will look at a simple regression model:
Recall the homoskedasticity assumption:
So we need the conditional variance (and thereby the unconditional) to not be a function of X.
If this assumption is violated, then our nice standard error formulas are no longer valid, and thus our procedure for hypothesis testing is also null.
So what do we do? Abandon all hope??
Pitfalls of OLS: heteroskedabbadoo
The estimate for can be expressed as .
This, again, leads to
Under homoskedasticity, the conditional variance becomes a constant, and the variance simplifies.
Under heteroskedasticity, the conditional variance is a function of X, and thus (potentially) differs for each observation.
We can rewrite the variance as
If we somehow know the functional form of the conditional variance, we can adjust our regression equation to avoid worrying about this. For now, let’s (realistically) assume we don’t know.
Estimating the variance
Now we have n different error variance terms to estimate. Before we had one, and it was quite simple to estimate. Now things are a bit more complicated.
It turns out that we can validly estimate the variance by replacing the individual error variances with the individual squared residuals:
This estimate of the variance is valid for heteroskedasticty of any form (and under homoskedasticity). Thus the resulting standard error (from taking the square root) is called the heteroskedasticity-robust standard error.
Robust inference
In the presence of heteroskedasticity, we can adapt our t statistics by simply replacing the standard errors with robust standard errors.
Modifying the formula for F statistics is a bit more complicated and not terribly informative for our purposes. In practice, you can conduct heteroskedasticity-robust inference in whatever statistical software package being used.
