Significant regression coefficient vs. significant overall model

[Lara333]

Hi everyone,

I have rather a simple question but for some reason have not been able to find a clear answer. My apologies if I have missed a previous answer in my searching.

Here is my question: What does it mean if you have a highly significant regression coefficient for a predictor, but overall the model is not significant due to the other non-significant predictors in the model? Do you just state that the model was non-significant and conclude you cannot predict the dependent variable, ignoring the significant relationship between your dependent measure and one of your predictors? This seems to be ignoring an interesting relationship in the data, but equally I realise that one can’t throw in every possible predictor under the sun and pick one that happens to be significant.

More specifically to my problem, I have a regression model containing 3 predictor variables (A, B and C) and all corresponding two-way and three-way interactions between them. The overall model is not significant as shown by the F test, but our three-way interaction term (which we a-priori predicted would be significant) is significant at p = .025. Can we interpret this or do we have to ignore the whole thing? And does the answer to this also therefore apply to the significance of the ‘corrected model’ in ANOVA too?

Sorry for the simplicity of the question, and thank you for any help you can give!

 

[Link]

If the overall model is not significant, I don't think you can trust the p-values for the independent variables as they tend to be unstable. How many observations are you working with? Can you state your full working model for me?

 

[Lara333]

Thanks for your reply!

My model contains the single terms A, B, C, the two-way interaction terms A*B, A*C, and B*C, and the three-way interaction term A*B*C. A and B are dichotomous categorical variables, and C is continuous. I have 69 observations.

The overall model is not significant, F(7,68) = .998, p = .442. However the A*B*C term is significant, beta = .705, t = 2.301, p = .025.

For this particular model, the overall significance is very poor and I didn’t really expect this to be interpretable, but was wondering what the rules are about this from a more general perspective, and whether these rules are the same for interpreting ANOVA output too.

One final related question: Sometimes I will calculate AIC of a full model and a reduced model, and the AIC values will be significantly lower for the reduced model than the full model, which I take to mean that this reduced model is better. However, the full model has overall significance from the F-test at p<.05, whereas the reduced model is overall not significant. What should be taken to decide which model to use? If I go by AIC values and present the reduced model, won’t reviewers then criticise me for my overall model not being significant? Sorry for the basic questions but I am very new to this area of analysis.

Many thanks!