Variance Inflation from Collinear Regressors

Isolating one regressor's independent variation

Regress $x_j$ on the remaining regressors and record $R_j^2$. The variance inflation factor is (O'Brien, 2007):

It compares the coefficient variance with its value under orthogonality, holding the residual variance and scale fixed.

The nonlinear increase

The increase accelerates as the auxiliary fit approaches one.

Assumptions and interpretation

VIF diagnoses the design matrix. It does not test omitted-variable bias or model validity. High collinearity can be a direct consequence of the research design, interactions, or polynomial terms.

Software differs on whether it includes the intercept and how it treats transformed terms. Exact aliasing makes the design matrix singular and the VIF undefined.

Geometrically, $x_j$ splits into a projection on the other regressors and an orthogonal remainder. The smaller remainder identifies its coefficient.

Interpretation and reporting

A VIF of $20$ means collinearity multiplies the coefficient variance by twenty under the comparison conditions. The corresponding standard-error multiplier is about $4.47$. It does not mean the coefficient itself is biased.

Report the auxiliary $R_j^2$, VIF, and regressor definition. Fixed cutoffs should not trigger automatic variable removal. Polynomial terms and interactions can have high VIF values because their construction creates dependence that the model requires.

Collinearity limits precision by leaving little independent variation in $x_j$. It does not diagnose omitted variables, reverse causality, or heteroskedasticity. Exact linear dependence is a separate problem because it prevents unique coefficient estimation.

Reproducible implementation

Use the regression's exact estimation sample and design transformations. For each nonintercept regressor, fit an auxiliary regression on all remaining columns with the intended intercept.

Calculate $1/(1-R_j^2)$ and compare it with the software output. Confirm that rescaling a regressor does not change its VIF. Detect aliased columns before interpreting extremely large values.

Inspect condition indices or singular values when dependence involves several columns at once. Center continuous variables before forming polynomials when that improves numerical interpretation. Preserve hierarchy when interactions are substantively required. Save the full column definitions so every auxiliary regression can be reconstructed (Belsley et al., 1980).

Further reading

O'Brien explains why fixed VIF thresholds have limited meaning (O'Brien, 2007). Belsley, Kuh, and Welsch discuss collinearity diagnostics based on the full design matrix (Belsley et al., 1980).