- Thread starter alexandros__23
- Start date

Theoretically, yes, but practically, not so much.

The significance of a regression, as per the usual F-test, is about how much the model reduces the variance of the DV. Multi-colinearity means that the DV is equally well-predicted by two or more IVs, but that doesn't make the DV any less well-predicted. Any linear combination of the multi-colinear IVs would produce an equally good fit, and the regression basically picks one at random. Removing multi-colinear IVs from the regression will cause it to pick a different linear combination for its fit, but it won't make the overall fit any better or worse.

How the multi-colinearity will show up is in the error bars on the regression coefficients. Since many linear combination produce an equally good fit, that means any given coefficient of the multi-colinear set can take on a wide range of values without detracting from the fit. Thus the error bars are wide. In fact, given multi-colinear data, it is possible to get a significant fit overall even though the confindence interval for every single regression coefficient includes zero.

Even though this story is conceptually right, it is still possible for multi-colinearity to affect the overall fit significance in unusual circumstances. The reason is that the F-test isn't purely about how much the variance was reduced, but by how much it was reduced per degreee of freedom. If you have very many multi-colinear IVs, it could occur that your F-test tells you that it expected a larger variance reduction than it got, given the large number of degrees of freedom in your fit. Then when you remove all but one of those multi-colinear IVs, the variance stays the same but the F-test is now satisfied with it, given the smaller number of degrees of freedom. In practice, I would only expect this to occur if a pretty big fraction of your IVs were multi-colinear, or if the F-test result is very much on the edge of significance.

The significance of a regression, as per the usual F-test, is about how much the model reduces the variance of the DV. Multi-colinearity means that the DV is equally well-predicted by two or more IVs, but that doesn't make the DV any less well-predicted. Any linear combination of the multi-colinear IVs would produce an equally good fit, and the regression basically picks one at random. Removing multi-colinear IVs from the regression will cause it to pick a different linear combination for its fit, but it won't make the overall fit any better or worse.

How the multi-colinearity will show up is in the error bars on the regression coefficients. Since many linear combination produce an equally good fit, that means any given coefficient of the multi-colinear set can take on a wide range of values without detracting from the fit. Thus the error bars are wide. In fact, given multi-colinear data, it is possible to get a significant fit overall even though the confindence interval for every single regression coefficient includes zero.

Even though this story is conceptually right, it is still possible for multi-colinearity to affect the overall fit significance in unusual circumstances. The reason is that the F-test isn't purely about how much the variance was reduced, but by how much it was reduced per degreee of freedom. If you have very many multi-colinear IVs, it could occur that your F-test tells you that it expected a larger variance reduction than it got, given the large number of degrees of freedom in your fit. Then when you remove all but one of those multi-colinear IVs, the variance stays the same but the F-test is now satisfied with it, given the smaller number of degrees of freedom. In practice, I would only expect this to occur if a pretty big fraction of your IVs were multi-colinear, or if the F-test result is very much on the edge of significance.

Last edited:

I did not understand anything from what you said because I do not know anything about statistics and econometrics. My question is related to a regression that is insignificant and which I am trying to explain in simple words why did tha happen. So, Can I say that maybe collinearity exist between IDV and as such it maybe possible fro the regression to be insignificant? Does that happen a lot in regression analyses? Can I check in some way that colinearity exist between my variables? That kind of answer I want to give?