Significant interaction effects but no significant simple effects

Ank

New Member
#1
Hi all!

For my thesis i'm having some trouble with interpreting a significant interaction effect while the simple effects are both insignificant.
I already checked related fora but in most cases the solution is, that it's a cross over interaction but i already checked this on my data and this is not the case.

I have a significant interaction between variable A (two levels) and variable B (two levels) on Y.
This would indicate the effect of A on Y differs by B, however the effect of A with level 1 or with level 2 are both insignificant.
Does anyone know how to interpret my data?

Thank you a lot!
 

noetsi

Fortran must die
#2
It could be a power issue. In ANOVA various tests have differing power and that matters in the statistical tests in ways that make little substantive sense. Multicolinearity could also be an issue although I am not sure how that impact simple effects.
 

Karabiner

TS Contributor
#3
There is no need that main effects are statistically significant when the interaction is statistically signficant.
See for example here https://i1.wp.com/statisticsbyjim.c...eractions_plot_categorical.png?resize=576,384

Just describing results in terms of signficant/nonsignificant is not very informative, by the way.
You should report the group sizes and the 4 means (with standard deviations), and the precise
p-values.

With kind regards

Karabiner
 

Ank

New Member
#4
@Karabiner i know the main effect does not need to be significant for the itneraction to be significant. However my problem is the simple effects are not significant while the interaction is. I already made a table like this:

Level 1 A Level 1 B (mean) (sd)
Level 2 A Level 1 B (mean) (sd) (p value)
Level 1 A Level 2 B (mean) (sd)
Level 2 A Level 2 B (mean) (sd) (p value)
 

Karabiner

TS Contributor
#7
It is hard to believe that the interaction effect was statistically significant.
Seemingly you have very large sample sizes?

With kind regards

Karabiner
 

Ank

New Member
#8
Yes my sample size consisted of around 20.000 observations. However, the interaction was marginally significant (0.054)
 

Karabiner

TS Contributor
#9
So you have a group difference of +0.025 standard deviations
in onesample, and of-0.05 standard deviations in the other sample.

Given the huge sample size, the standard errors for these differences
will be tiny, therefore you can interpret them straightforward, I suppose.

Are effect sizes between 0.025 and -0.05 standard deviations worth
cosidering in the context of your study and your research questions?

With kind regards

Karabiner
 

noetsi

Fortran must die
#10
With a large enough sample size lots of things are statistically significant. The question is if they are substantially significant (effect size). I don't know how to interpret the effect size of interaction, you either have it or you don't.