Multiple regression- model non-significant, but a significant predictor?

#1
Hi,

I'm currently analyzing the results for my final year dissertation. For one of my multiple regressions, the overall regression model is non-significant (.17) with a very small adjusted R square of .03. Whilst most of my predictors are non-significant, I have one significant predictor (an interaction). When I asked my supervisor whether such a predictor is worth looking into (i.e plotting a graph to understand the interaction) he seemed to think not, and suggested just briefly mentioning the interaction.

However I've since read that it's actually wrong to ignore it, on the basis of a non-significant overall model?

Thanks for any advice!
 

rogojel

TS Contributor
#2
hi, you could try to build a reduced model, with only the interaction and its component factors and see what you get. Unfortunately the very small r-squared is not a good sign...

regards
 

noetsi

Fortran must die
#3
I have never read that it makes sense to talk about any variable in a model which is not significant. Also that is a really small r square.

I always thought that listening to you major professor is a good idea in a dissertation :p He or she likely knows what is acceptable in the profession a lot better than you do.
 
#4
just to chime in

r squared needs to really be interpreted in the context of existing literature on what you are studying. To give an example- you are studying a phenomena where you know that there exists a covariate (or two) that explains 85% of the phenomena (lets say odds ratio of entering the NBA and height + athletic ability). You want to study how self-regulation affects this odds ratio. Any model you create without the covariates will likely have minimal r-squared values when applied to a randomly selected sample. BUT what you need to do is look at the remaining r-squared and based your assessment from that. So if your if your model explains 3% of the remaining 15% variance, thats a sizable chunk. So what separates tall athletic people in the NBA and those not can be attributed to self-regulation by about 20%.

There are three take aways from this when looking at r-squared-
1. what does the literature say
2. what covariates mights exist that affect your model
3. what kind of population are you testing on the model (i.e. does it make sense for that population, i.e. testing how self-regulation in a random individual affects their likelihood on entering the NBA is not a meaningful test due to the presence of a severe covariate)
 

CB

Super Moderator
#5
I have never read that it makes sense to talk about any variable in a model which is not significant.
Possibly I'm interpreting "talk about" in a different way than you mean, but I'm sure one might sometimes want to say things about an effect that isn't significant? People say all kinds of things about all kind of topics :yup:

For one of my multiple regressions, the overall regression model is non-significant (.17) with a very small adjusted R square of .03. Whilst most of my predictors are non-significant, I have one significant predictor (an interaction). When I asked my supervisor whether such a predictor is worth looking into (i.e plotting a graph to understand the interaction) he seemed to think not, and suggested just briefly mentioning the interaction.
I'd say that the non-significant overall model and low R squared may be useful pieces of information to provide, but they imply pretty much nothing about what you should choose to actually write about in your report. Your study presumably set out to answer some questions: Give the necessary information to answer those questions. If you predicted that a particular effect would be statistically significant, and it isn't, say that. If you predicted that this interaction would be significant and it is, say so.

However, if you did not specify any hypotheses about this interaction, and don't have any pre-existing reasons to expect it to be important, then don't make a big deal about this interaction or start trying to put together an explanations for why it might be there. Chances are that the interaction may just be noise. :)
 

rogojel

TS Contributor
#6
I have never read that it makes sense to talk about any variable in a model which is not significant.
Hi noetsi,
for instance Minitab will give a warning about a non-hierarchical model IIRC if I do not include both terms together with the interaction, even if the terms are not significant. The reason I found is that the model will not be invariant to a change of the zero point of IVs unless I have both variables in it.