GLMM: Main effect cancels interaction effect

#1
Dear all,
I am running a GLMM with a 3-way interaction.
When I enter the 3rd variable of Age (young vs old) only within the 3-way intraction, it is highly significant. BUT when I also enter Age as a main effect, the interaction disappears.
For clarity, when I run separate GLMMs for young and old, the 2-way interaction (RI*ISI) appears only for the old, not for the young, so theoretically the 3-way interaction exists, though it is very small.

My question:
WHY does adding Age as a main effect remove the 3-way interaction.
ANY help will be greatly appreciated. A link, a term to look up, anything please.

Thanks in advance.

Bonus question: Is it possible to calculate effect size (e.g. cohen's d) from the output of a GLMM?
With&WithoutAge.PNG
 

hlsmith

Less is more. Stay pure. Stay poor.
#2
Well I will start by saying that people believe interactions in real practice are rare - let alone a three-way interaction. It is standard practice to include the base terms in the model (e.g., so age). The models usually also do better when the variables are centered.

Can you theoretically justify a three-way interaction? So think about what it is saying and believe that this just isn't a spurious finding? Also, interactions will suck up degrees of freedom and require larger sample sizes. If you think that running the two way interactions in the two stratified samples by the third variable - I would plot both of those models with confidence intervals looking for actual disordinal effects that actually seem real and practically relevant. This may also help you understand why the above saturated model is null for the term.

Also, when you say GLMM you are referencing mixed (multi-level) modeling. If so, what do the full sample look like and what is your sample size?
 
#3
Thank you so much for replying!!

In answer to your questions:
- The interaction could make sense theoretically. It is a known cognitive interaction and pre-pubescent children's brains haven't yet developed sufficiently for this to occur. However, the difference in scores is tiny and the CIs are mostly overlapping (see image). That said, when looking at each age group individually (13, 14, 15, 16 etc.) the interaction is always there, while it is always absent from the younger ages. It could make sense either way.

-The model is 117 participants tested on 16 test items each, making 1872 trials. Participant and Item are treated as random effects.
ISI - within subjects, 2 levels (8 items, 8 items)
RI - between subjects, 2 levels

Could you explain to me (or point me in the right direction) how adding one variable (e.g. age) affects another (e.g. interaction) in the model? Is it that the Age effect, without the Age variable, causes the low p value in the interaction variable because it is actually accounting for the large main effect of Age? Or is it the case that all variables interact with each other within the model?

There is another, similar variable I am checking instead of age, whereby the 3-way interaction is significant with and without the main effect, and more clearly so, but the F and p values are slightly different, so I'm just trying to understand what it means and how it works.

Thanks again!

Age RI ISI interaction..PNG