Multicollinearity

Regressors are perfectly collinear when one is an exact linear combination of the others (and the constant). Examples: x₁ = 5x₂ (a variable entered twice in different units), a dummy for every category with a constant (lesson 5), or an age, a year of birth and a survey year. Then X′X is singular, (X′X)⁻¹ does not exist, and the normal equations have infinitely many solutions. Software drops a column or refuses to run.

Not everything is lost. Take y = a + b₁K + b₂L + b₃(K + L) + u. Substituting, y = a + (b₁ + b₃)K + (b₂ + b₃)L + u, so the regression can estimate b₁ + b₃ and b₂ + b₃ but never b₁, b₂ and b₃ separately: three numbers are asked of data that only contain two. The identifiable quantities are the combinations that survive the substitution. The same holds for any exact relation: whatever you can express without the redundant variable is estimable, and nothing else is.

Perfect collinearity is a failure of the model (a violated assumption of lesson 2), not of the sample: no amount of data fixes it. It is why an examiner who writes “X₁ = 5X₂” is testing the assumption “no perfect multicollinearity”, not homoscedasticity or linearity.

In practice regressors are rarely perfectly related, but they are often highly related: income and wealth, schooling and experience, GDP and investment. The regression then runs and gives numbers, but the numbers are uncertain. Lesson 1 showed that a coefficient is the slope on the part of x₁ that the other regressors do not explain. The more the others explain, the less is left, and the less is left to learn from:

In the lab, two regressors each have a true coefficient of 1. Each press draws 500 samples and plots the pair of estimates from every one. At ρ = 0 the cloud is round and tight. Raise ρ and it stretches into a long thin ellipse along the line where the estimates add to 2. Notice three things: the centre stays on (1, 1), so nothing is biased; the spread of each estimate balloons; and the spread of the sum hardly moves.

The classic symptoms, and what each really tells you:

• A high R² and a significant F, with few significant t ratios. The regressors jointly explain y but the regression cannot assign credit among them. This is the textbook sign (the opening question). The reason a joint test can reject when each t does not is lesson 3’s confidence ellipse: the long thin cloud above is inside the box of two intervals but far from the truth along its narrow side.
• Large standard errors and unstable coefficients. Small changes in the data, or in the list of regressors, move the estimates a lot, and signs can flip.
• High pairwise correlations among regressors. Sufficient to worry, but not necessary: the correlation matrix can be mild while a three-way relationship is strong.
• A large VIF or a small tolerance. The usual rule of thumb: VIF above 10 (R₁² above 0.9, TOL below 0.1) signals serious collinearity. The closer TOL is to 1, the less of a problem.

Other tools you will see named: Klein’s rule of thumb (collinearity is a problem if an auxiliary R² exceeds the R² of the main regression), the Farrar-Glauber tests, and the condition number of X′X. They all measure the same thing from different angles.

A real example. Life expectancy in 186 economies in 2019 on ln GDP per head, health spending, the urban share and the fertility rate (World Bank). The variables are related: fertility and ln GDP per head have a correlation of −0.82, and urbanisation and ln GDP of 0.69. The regression has R² = 0.77 and F = 151.9, and the VIFs are:

Nothing is near 10. This is typical of real data: collinearity is common, but a VIF over 10 is rarer than the rule’s fame suggests. Dropping the fertility rate lowers the standard error of the ln GDP coefficient from 0.532 to 0.39 and raises its estimate from 3.31 to 5.9, which shows both sides of the trade: less collinearity, and a coefficient that now also carries fertility’s effect (lesson 7).

Hence the exam answers: with high but not perfect multicollinearity, estimates may have large standard errors, wide confidence intervals, a high R² and few significant coefficients, but they are all still BLUE, and “almost all the estimates are statistically significant” is the one thing that does not follow. And “coefficients are estimated with bias” is not an indicator of multicollinearity.

• Do nothing if the aim is prediction, or the effect of the group of variables together. The problem is only the split between them, which may not matter.
• Get more data, or more varied data. More observations shrink every variance; a sample where the regressors move independently is better still.
• Combine the variables into one (an index or a ratio) when theory says they measure one thing, or impose a restriction from theory (equal coefficients, constant returns) and test it with an F test (lesson 3).
• Drop one variable, with care. It lowers the variance but, if the dropped variable truly belongs, it biases what is left (lesson 7): the opening question’s last statement. It is a trade of bias for variance, not a free fix.
• Use a shrinkage method such as ridge regression, which accepts a little bias for a large cut in variance. It sits outside the usual entrance syllabus.

The Delhi School’s questions here (an equation with a redundant combination of regressors, the indicators of multicollinearity, and the opening puzzle) are in the hard-problems lesson at the end of the module. The Farrar-Glauber and Klein matching items are in the test atlas.