looking for a tutor for Econometrics & Statistics
John Jay
CH102.pdf
1
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 1
• Assumption 8 of the classical linear regression model (CLRM) is that there is no multicollinearity among the regressors included in the regression model.
• We shall take a critical look at this assumption by seeking answers to the following questions:
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 2
1. What is the nature of multicollinearity?
2. Is multicollinearity really a problem?
3. What are its practical consequences?
4. How does one detect it?
5. What remedial measures can be taken to alleviate the problem of multicollinearity?
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 3
The Nature of Multicollinearity
• Originally, the term multicollinearity meant the existence of a “perfect” or exact linear relationship among some or all explanatory variables of a regression model.
• Assume the k–variable regression involving explanatory variables X1, X2,… Xk, (where X1 = 1 for all observations to allow for the intercept term).
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
2
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 4
The Nature of Multicollinearity
• An exact linear relationship is said to exist if the following condition is satisfied:
where l1, l2,… lk, are constants such that not all of them are zero simultaneously.
0 ... 2211
kk
XXX lll (10.1.1)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 5
The Nature of Multicollinearity
• Today, however, the term multicollinearity is used in a broader sense to include the case of perfect multicollinearity, as well as the case where the X variables are intercorrelated but not perfectly so, as follows:
0 ... 2211
ikk
vXXX lll (10.1.2)
where vi is a stochastic error term.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 6
The Nature of Multicollinearity
• To see the difference between perfect and less than perfect multicollinearity, assume for example, that l20. Then (10.1.1) can be written as
(10.1.3)
which shows how X2 is exactly linearly related to the other variables or how it can be derived from a linear combination of the other X variables.
ki
k
iii XXXX
2
3
2
3
1
2
1
2 ... l l
l l
l l
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
3
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 7
The Nature of Multicollinearity
(10.1.4)
which shows that X2 is not an exact linear combination of the other X’s because it is also determined by the stochastic error term vi .
iki
k
iii vXXXX
22
3
2
3
1
2
1
2
1 ... ll
l l l
l l
• Similarly, if l20, Eq. (10.1.2) can be written as
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 8
The Nature of Multicollinearity
• Consider the following data:
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 9
The Nature of Multicollinearity
• It is apparent X3i = 5X2i. Therefore, there is perfect collinearity between X2 and X3 since r23 = 1.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
4
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 10
The Nature of Multicollinearity
• X3* was derived from X3 by adding the random numbers 2, 0, 7, 9, 2. There is no longer perfect collinearity between X2 and X3* but the correlation coefficient is 0.9959.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 11
FIGURE 10.1: The Ballentine view of multicollinearity
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 12
The Nature of Multicollinearity
• There are several sources of multicollinearity:
1. The data collection method employed
2. Constraints on the model or in the population being sampled
3. Model specification
4. An overdetermined model.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
5
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 13
Estimation in the Presence of Perfect Multicollinearity
• Using the deviation form, we can write the three– variable regression model as
iiii uxxy ˆ ˆ ˆ
3322 (10.2.1)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 14
Estimation in the Presence of Perfect Multicollinearity
• From Chapter 7 we obtain
(7.4.7)
2
32
2
3
2
2
323
2
32
2 )( ))((
))(( ))(( ˆ
iiii
iiiiiii
xxxx
xxxyxxy
2
32
2
3
2
2
322
2
23
3 )( ))((
))(( ))(( ˆ
iiii
iiiiiii
xxxx
xxxyxxy (7.4.8)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 15
Estimation in the Presence of Perfect Multicollinearity
• Assume that X3i = lX2i, where l is a nonzero constant (e.g., 2, 4, 1.8, etc.). Substituting this into (7.4.7) we obtain
0 0
)( ))(( ))(( ))((
ˆ 22
2
22
2
22
2
2
22
2
2
2
2
2
iii
iiiiii
xxx xxyxxy
ll lll
(10.2.2)
which is an indeterminate expression.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
6
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 16
Estimation in the Presence of Perfect Multicollinearity
• For practical purposes, X2 and X3 are indistinguishable.
• In applied econometrics this problem is most damaging since the entire intent is to separate the partial effects of each X upon the dependent variable.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 17
Estimation in the Presence of Perfect Multicollinearity
• To see this differently, let us substitute X3i = lX2i, into (10.2.1) and obtain the following:
(10.2.3)
where
ii
ii
iiii
ux
ux
uxxy
ˆ ˆ
ˆ )ˆ ˆ(
ˆ )(ˆ ˆ
2
232
2322
l
l
)ˆ ˆ( ˆ 32
l (10.2.4)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 18
Estimation in the Presence of Perfect Multicollinearity
• Applying the usual OLS formula to (10.2.3), we get
(10.2.5)
• Therefore, although we can estimate uniquely, there is no way to estimate 2 and 3 uniquely.
2
2
2
32 )ˆ ˆ( ˆ
i
ii
x
yxl
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
7
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 19
Estimation in the Presence of Perfect Multicollinearity
• Mathematically,
(10.2.6)
• This gives us only one equation and two unknowns (note l is given) with an infinity of solutions.
32 ˆ ˆ ˆ l
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 20
Estimation in the Presence of “High” but “Imperfect” Multicollinearity
• Turning to the three–variable regression model given in (10.2.1),
(10.2.1)
instead of exact multicollinearity we may have
iiii uxxy ˆ ˆ ˆ
3322
iii vxx
23 l (10.3.1)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 21
Estimation in the Presence of “High” but “Imperfect” Multicollinearity
iii vxx
23 l (10.3.1)
where l0 and where vi is a stochastic error term such that Sx2ivi = 0.
• In this case, estimation of the regression coefficients 2 and 3 may be possible.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
8
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 22
Estimation in the Presence of “High” but “Imperfect” Multicollinearity
• For example, substituting (10.3.1) into (7.4.7), we obtain
where use is made of Sx2ivi = 0.
22
2
222
2
22
2
2
22
22
2
2
2
2 )( ))((
))(( ))(( ˆ
iiii
iiiiiiiii
xvxx
xvyxyvxxy
ll lll
(10.3.2)
• A similar expression can be derived for . 3
̂
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 23
Multicollinearity: Much Ado About Nothing? Theoretical Consequences of Multicollinearity
• First, it is true that even in the case of near multicollinearity the OLS estimators are unbiased.
• Second, it is also true the collinearity does not destroy the property of minimum variance.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 24
Multicollinearity: Much Ado About Nothing? Theoretical Consequences of Multicollinearity
• Third, multicollinearity is essentially a sample (regression) phenomenon in the sense that even if the X variables are not linearly related in the population, they may be so related in the particular sample at hand.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
9
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 25
Practical Consequences of Multicollinearity
1. Although BLUE, the OLS estimators have large variances and covariances, making precise estimation difficult.
2. Because of consequence (1), the confidence intervals tend to be much wider, leading to the acceptance of the “zero null hypothesis” more readily.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 26
Practical Consequences of Multicollinearity
3. Also because of consequence (1), the t ratio of one or more coefficients tends to be statistically insignificant.
4. Although the t ratio of one or more coefficients is statistically insignificant, R2, the overall measure of goodness of it, can be very high.
5. The OLS estimators and their standard errors can be sensitive to small changes in the data.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 27
Practical Consequences of Multicollinearity
• Recall that for the model (10.2.1) the variances of the slopes are given by
Large Variances and Covariances of OLS Estimators
) 1( )ˆvar(
2
3 2
2
2
2
2 rx i
) 1( )ˆvar(
2
3 2
2
3
2
3 rx i
(7.4.12)
(7.4.15)
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
10
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 28
Practical Consequences of Multicollinearity
• The covariance is given by
Large Variances and Covariances of OLS Estimators
(7.4.17) ) 1(
)ˆ ,ˆcov( 2
3
2
2
2
3 2
2
3 2
32
ii
xxr
r
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 29
Practical Consequences of Multicollinearity
• The speed with which variances and covariances increase can be seen with the variance–inflation factor (VIF), which is defined as
Large Variances and Covariances of OLS Estimators
) 1( 1
VIF 2
3 2 r
(10.5.1)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 30
Practical Consequences of Multicollinearity
Large Variances and Covariances of OLS Estimators
) 1( 1
VIF 2
3 2 r
(10.5.1)
• As r23 approaches 1, the VIF approaches infinity.
• If there is no collinearity between X2 and X3 the VIF will be ?
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
11
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 31
Practical Consequences of Multicollinearity
• Using this definition, we can express (7.4.12) and (7.4.15) as
Large Variances and Covariances of OLS Estimators
(10.5.2)
(10.5.3)
VIF )ˆvar( 2
2
2
2
i x
VIF )ˆvar( 2
3
2
3
i x
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 32
TABLE 10.1: THE EFFECT OF INCREASING r23 ON VAR AND COVAR)ˆ( 2 )ˆ ,ˆ( 32
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 33
FIGURE 10.2: The behavior of var as a function of r23)ˆ( 2
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
12
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 34
Practical Consequences of Multicollinearity
• Before proceeding further, it may be noted that the inverse of VIF is called tolerance (TOL). That is,
Large Variances and Covariances of OLS Estimators
)(1 VIF
1 TOL 2
j
j
j R (10.5.5)
• With perfect collinearity, TOLj=0, and with zero collinearity TOLj =1.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 35
Practical Consequences of Multicollinearity
• Because of the large standard errors, the confidence intervals for the relevant population parameters tend to be larger.
Wider Confidence Intervals
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 36
TABLE 10.2: THE EFFECT OF INCREASING COLLINEARITY ON THE 95% CONFIDENCE INTERVAL FOR 2 : )ˆse( 96.1 ˆ 22
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
13
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 37
Practical Consequences of Multicollinearity
• Recall that to test the null hypothesis that, say 2 0, we use the t ratio
“Insignificant” t Ratios
and compare the estimated t value with the critical t value from the t table.
)ˆ/se(ˆ 22
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 38
Practical Consequences of Multicollinearity
• But as we have seen, in cases of high collinearity, the estimated standard errors increase dramatically, thereby making the t values smaller.
“Insignificant” t Ratios
• Therefore, in such cases, one will increasingly accept the null hypothesis that the relevant true population value is zero.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 39
Practical Consequences of Multicollinearity
• Consider the k–variable linear regression model:
A High R2 but Few Significant t Ratios
• One or more of the partial slope coefficients are individually statistically insignificant on the basis of the t test.
ikikiii uXXXY ...
33221
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
14
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 40
Practical Consequences of Multicollinearity
• The R2 in such situations may be so high, say in excess of 0.9, that on the basis of the F test one can convincingly reject the hypothesis that
A High R2 but Few Significant t Ratios
0 ... 32
k
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 41
Practical Consequences of Multicollinearity
Sensitivity of OLS Estimators and Their Standard Errors to Small Changes in Data
TABLE 10.3 TABLE 10.4 HYPOTHETICAL DATA ON Y, X2 and X3 HYPOTHETICAL DATA ON Y, X2 and X3
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 42
Practical Consequences of Multicollinearity
Regression based on Table 10.3:
2 df 0.00868 )ˆ ,ˆcov(
0.5523 0.8101
(0.0358) (2.4151) (1.5431)
(0.0851) (0.1848) (0.7737)
0.0030 0.4463 1.1939 ˆ
32
3 2
2
32
rR
t
XXY iii
(10.5.6)
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
15
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 43
Practical Consequences of Multicollinearity
Regression based on Table 10.4:
(10.5.7)
2 df 0.0282 )ˆ ,ˆcov(
0.8285 0.8143
(0.2158) (1.4752) (1.6187)
(0.1252) (0.2721) (0.7480)
0.0270 0.4014 1.2108 ˆ
32
23
2
32
rR
t
XXY iii
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 44
An Illustrative Example
TABLE 10.5: HYPOTHETICAL DATA ON CONSUMPTION EXPENDITURE Y, INCOME X2 , AND WEALTH X3
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 45
An Illustrative Example
• From Table 10.5, we obtain the following regression:
7df 9531.0 9635.0
)5261.0( )1442.1( )6690.3(
)0807.0( )8229.0( )7525.6(
0424.0 9415.0 7747.24 ˆ
22
32
RR
t
XXY iii
(10.6.1)
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
16
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 46
An Illustrative Example
TABLE 10.6: ANOVA TABLE FOR THE CONSUMPTION– INCOME–WEALTH EXAMPLE
• Under the usual assumption we obtain
4019.92 3494.46
7770.282,4 F (10.6.2)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 47
FIGURE 10.3: Individual confidence intervals for 2 and 3 and joint confidence interval (ellipse) for 2 and 3.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 48
An Illustrative Example
• If we regress X3 on X2 we obtain
(10.6.3)
9979.0 )0405.62( )2560.0(
)1643.0( )4758.29(
9415.0 5454.7 ˆ
2
23
Rt
XX ii
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
17
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 49
An Illustrative Example
• Now let us see what happens if we regress Y on X2 only:
(10.6.4)
9621.0 )2432.14( )8128.3(
)0357.0( )4138.6(
5091.0 4545.24 ˆ
2
2
Rt
XY ii
• In (10.6.1) the income variable was statistically insignificant, but now it is highly significant.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 50
An Illustrative Example
• If we regress Y it on X3 we obtain:
(10.6.5)
• Wealth has now a significant impact on consumption expenditure, whereas in (10.6.1) it had no effect.
9567.0 )29.13( )551.3(
)0047.0( )874.6(
0498.0 411.24 ˆ
2
3
Rt
XY ii
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 51
An Illustrative Example
TABLE 10.7: U.S. CONSUMPTION FOR THE 1947-2000
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
18
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 52
An Illustrative Example
• We use the following for analysis:
(10.6.6)
where: Yd = Real disposable personal income W = Real wealth I = Real interest rate
ttttt uIWYdC
4321 lnlnln
• The results of regression (10.6.6) are as follows:
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 53
An Illustrative Example
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 54
Detection of Multicollinearity
1. High R2 but few significant t ratios. As noted, this is the “classic” symptom of multicollinearity.
The disadvantage of this criterion is that “it is too strong in the sense that multicollinearity is considered as harmful only when all of the influences of the explanatory variables on Y cannot be disentangled”.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
19
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 55
Detection of Multicollinearity
2. High pair–wise correlations among regressors. If the pair–wise coefficient between two regressors is high, say, in excess of 0.8, then multicollinearity is a serious problem.
The problem with this criterion is that, although high zero–order correlations may suggest collinearity, it is not necessary that they be high to have collinearity in any specific case.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 56
Detection of Multicollinearity
High zero–order correlations are a sufficient but not necessary condition for the existence of multicollinearity.
This is because because it can even exist even though the zero–order or simple correlations are comparatively low (say, less than 0.50).
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 57
Detection of Multicollinearity
4. Auxiliary regressions. Multicollinearity can arise because one or more of the regressors are exact or approximately linear combinations of the other regressors.
One way of finding out which X variable is related to other X variables is to regress each Xi on the remaining X variables and compute the corresponding R2, which we designate as .2
i R
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
20
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 58
Detection of Multicollinearity
Each of these regressions is called an auxiliary regression.
Then, following the relationship R2 between F and established in (8.4.11), the variable
)1/()1(
)2/( 2
.
2
.
32
32
knR
kR F
ki
ki
xxxx
xxxx
i
(10.7.3)
follows the F distribution with k – 2 and n – k + 1 df.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 59
Detection of Multicollinearity
If the computed F exceeds the critical Fi at the chosen level of significance, it means that the particular Xi is collinear with the other X’s.
This method may not work if there are several complex linear associations among the independent variables.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 60
Detection of Multicollinearity
Instead of formally testing all auxiliary R2 values, one may adopt Klein’s rule of thumb.
The rule suggests that multicollinearity may be a troublesome problem only if the R2 obtained from an auxilary regression is greater than the overall R2 that is obtained from the regression of Y on all the regressors.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
21
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 61
Detection of Multicollinearity
6. Tolerance and variance inflation factor. We have already introduced TOL and VIF.
As the coefficient of determination in the regression of regressor Xj on the remaining regressors in the model, increases toward unity, VIF also increases and in the limit it can be infinite.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 62
Detection of Multicollinearity
As a rule of thumb, if the VIF of a variable exceeds 10, which will happen if exceeds 0.90, that variable is said to be highly collinear.
Of course, we could use TOLj as a measure of multicollinearity in view of its intimate connection with VIFj.
2 jR
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 63
Detection of Multicollinearity
The closer is TOLj to zero, the greater the degree of collinearity of that variable with the other regressors.
The closer TOLj is to 1, the greater the evidence that Xj is not collinear with the other regressors.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
22
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 64
Detection of Multicollinearity
7. Scatterplot. It is good practice to use a scatterplot to see how the various variables in a regression model are related.
Figure 10.4 presents the scatterplot for the U.S. consumption example discussed in the previous section.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 65
Detection of Multicollinearity
This is a four–by–four diagram.
We have four variables in the model, a dependent variable (C) and three explanatory variables:
Real disposable income (Yd) Real wealth (W) Real interest rate (I).
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 66
FIGURE 10.4: Scatterplot for Example 10.2 data
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
23
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 67
Remedial Measures
• If multicollinearity is serious we have two choices:
(1) Do nothing
(2) Follow some rules of thumb.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 68
Remedial Measures
• Multicollinearity is God’s will.
Do nothing
• It is a data deficiency problem.
• Not all variables may be insignificant.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 69
Remedial Measures
1. A priori information. Consider the model:
Rule-of-Thumb Procedures
iiii uXXY
33221
where Y = consumption, X2 = income, and X3 = wealth, which are expected to be highly collinear.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
24
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 70
Remedial Measures
Rule-of-Thumb Procedures
iiii uXXY
33221
But suppose a priori we believe that 3 = 0.102.
We then run the following regression:
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 71
Remedial Measures
Rule-of-Thumb Procedures
where
ii
iiii
uX
uXXY
21
32221 10.0
iii XXX
32 1.0
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 72
Remedial Measures
2. Combining cross–sectional and time series data. Suppose we want to study the demand for automobiles in the United States.
Rule-of-Thumb Procedures
Assume we have time series data on the number of cars sold, average price of the car, and consumer income.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
25
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 73
Remedial Measures
Suppose also that
Rule-of-Thumb Procedures
where Y = number of cars sold, P = average price, I = income, and t = time. In time series data, the price and income variables tend to be highly collinear.
tttt uIPY lnlnln
321
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 74
Remedial Measures
A way of this has been suggested by Tobin.
Rule-of-Thumb Procedures
If we have cross–sectional data we can obtain an estimate of the income elasticity 3 because in such data, which are at a point in time, the prices do not vary much.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 75
Remedial Measures
Rule-of-Thumb Procedures
We then write the preceding time series regression as
Let us call this cross–sectionally estimated income elasticity .
3 ̂
ttt uPY ln
21
where .IYY t
lnˆln 3
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
26
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 76
Remedial Measures
Rule-of-Thumb Procedures
ttt uPY ln
21
where .IYY t
lnˆln 3
Y* represents the value of Y after removing from it the effect of income. We can now obtain an estimate of the price elasticity 2 from the preceding regression.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 77
Remedial Measures
3. Dropping a variable(s) and specification bias. When faced with severe multicollinearity, one of the “simplest” things to do is to drop one of the collinear variables.
Rule-of-Thumb Procedures
But in dropping a variable from the model we may be committing a specification bias or specification error.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 78
Remedial Measures
4. Transformation of variables. Suppose we have time series data on consumption expenditure, income, and wealth.
Rule-of-Thumb Procedures
One way of minimizing the dependence between income and wealth is to proceed as follows.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
27
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 79
Remedial Measures
If the relation
Rule-of-Thumb Procedures
holds at time t, it must also hold at time t–1 because the origin of time is arbitrary anyway. Therefore, we have
tttt uXXY
33221
11 ,331 ,2211 tttt uXXY
(10.8.3)
(10.8.4)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 80
Remedial Measures
If we subtract (10.8.4) from (10.8.3), we obtain
Rule-of-Thumb Procedures
where .
(10.8.5)ttttttt vXXXXYY )()( 1 ,3331 ,2221
1
ttt uuv
Equation (10.8.5) is known as the first difference form.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 81
Remedial Measures
Another commonly used transformation in practice is the ratio transformation. Consider the model:
Rule-of-Thumb Procedures
(10.8.6)
where Y is consumption expenditure in real dollars, X2 is GDP, and X3 is total population.
tttt uXXY
33221
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
28
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 82
Remedial Measures
Since GDP and population grow over time, they are likely to be correlated. One “solution” to this problem is to express the model on a per capita basis:
Rule-of-Thumb Procedures
(10.8.7)
t
t
t
t
tt
t
X
u
X
X
XX
Y
3
3
3
2
2
3
1
3
1
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 83
Remedial Measures
5. Additional or new data. Since multicollinearity is a sample feature, it is possible that in another sample involving the same variables collinearity may not be so serious as in the first sample.
Rule-of-Thumb Procedures
Sometimes simply increasing the size of the sample (if possible) may moderate the collinearity problem.
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 84
Remedial Measures
In the three–variable model we saw that
Rule-of-Thumb Procedures
As the sample size increases, the sum of squared deviations of X will generally increase (Why?)
)1( )ˆvar(
2
3 2
2
2
2
2 rx i
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
29
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 85
Remedial Measures
As an illustration, consider the following regression of consumption expenditure Y on income X2 and wealth X3 based on 10 observations and 40 observations, respectively.
Rule-of-Thumb Procedures
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 86
Remedial Measures
9682.0 )1595.1( )7726.2( )875.3(
0349.0 8716.0 377.24 ˆ
2
32
Rt
XXY iii
9672.0 )0014.2( )0014.6( )8713.0(
06059.0 7299.0 0907.2 ˆ
2
32
Rt
XXY iii
10 observations:
40 observations:
(10.8.8)
(10.8.9)
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 87
Remedial Measures
6. Reducing collinearity in polynomial regressions. In these models the explanatory variable(s) appear with various powers.
Rule-of-Thumb Procedures
Thus, in the total cubic cost function involving the regression of total cost on output, (output)2 and (output)3, the various output terms will be correlated.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
30
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 88
Remedial Measures
In practice though, it has been found that if the explanatory variable(s) are expressed in deviation form, multicollinearity is substantially reduced.
Rule-of-Thumb Procedures
CHAPTER 10: Multicollinearity: What Happens If the Regressors are Correlated?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 89
Remedial Measures
Rule-of-Thumb Procedures
7. Other methods of reducing multicollinearity. Multivariate statistical techniques such as factor analysis and principal components or techniques such as ridge regression are often employed to “solve” the problem of multicollinearity.
ECO 382 CH 10 HANDOUT - DR. ANDREW PAIZIS
