Multiple Regression - Multicollinearity

#1
Hi I'd really appreciate it if anyone could help me with this.

I'm doing statistics coursework as part of a Psychology degree and have come across an issue regarding multicollinearity. My coursemates have informed me that this can be reduced/eliminated by using a backwards Multiple Regression rather than a Stepwise Multiple Regression. However I can find no reference for this technique in any books! Does anyone know if this is a valid method?

Thank you!
 
#2
Argh! Anyone who claims they have a technique to reduce/eliminate multicolinearity needs to have a heavy statistics textbook dropped on them.

Multicolinearity is a real aspect of the data. It tells you that the data do not distinguish which of two or more factors predicts the dependent variable, because those factors are too closely correlated for it to do so. This may not be the answer you wanted to get, but it is a real answer and any technique to make it go away is called "cheating".
 

bugman

Super Moderator
#3
ichbin is right. But I think the terminology used such as elminate or reduce are slighly misleading. Using stepwise models and model selection tools such as AIC and BIC will help you decide which model best describes your data. Also, another way to address this is through intial data exploration and looking at which varaibles are correlated.

As an example, in my work we often find alkalinity and pH are highly correlated. We usually drop alkalinity from any formal analyses becasue pH is more biologically meaningful. So if you start by examining your data and making dome informed judgements, this is a good step to resolving the issue. If you cannot drop terms, then try stepwise models with model selection tools.
 
#4
aha yeah I thought what they said sounded wrong which is why I wanted to check on here, I'm glad I did!. Thank you very much for both your answers, they were very useful!
 
#5
Question about multicollinearity: If you have two variables that are collinear (e.g. month and average monthly precipitation), would you keep the one that's more biologically meaningful (precipitation), or the one that allows for a higher AIC score in the model (month)?
 

bugman

Super Moderator
#6
@jeskill,

I would need to know a bit more about your data, variables and what you are trying to do.

Month is also meanginful in the biological sense. Think about seasonal patterns. These may not just be driven by rain, but temperature, day length ...