Generalized linear mixed model: Type III significant but multiple comparisons not significant

[Flangie]

Hi guys

I am running a glmm model for my data, the Type III tests of fixed effects is showing that there is a significant effect of treatment but when i run the multiple comparisons, non of the treatment levels are significant. What causes this confusion? Is my model not right maybe?

Regards,
Flangie

 

[Dason]

Imagine you're teaching at the front of class writing on the blackboard and suddenly you get hit with a spitball. You turn around and see the spitball on the ground. The direction it hit you from indicated it was some student. But you can't be sure which one.

So you have enough evidence to convince yourself that someone is guilty but you don't have enough to figure out who is guilty.

You basically have the same thing going on.

 

[Flangie]

Imagine you're teaching at the front of class writing on the blackboard and suddenly you get hit with a spitball. You turn around and see the spitball on the ground. The direction it hit you from indicated it was some student. But you can't be sure which one.

So you have enough evidence to convince yourself that someone is guilty but you don't have enough to figure out who is guilty.

You basically have the same thing going on.

Exactly the same situation. a very frustrating situation i wont wish to anyone. Though i hope someone has encountered that and might be helpful.

 

[Miner]

Another issue is that when you run multiple comparisons, the test (say Tukey's) places a fairly high hurdle to pass to demonstrate significance. This is because when you test all possible combinations of treatments you increase the risk of a false positive. If you were to have an a priori theory as to a much smaller set of combinations (say 1 or 2) that might explain the differences, you can limit the number of pairwise comparisons and have a lower hurdle to pass. It is important that this be an a priori theory and not the result of data snooping and testing the min/max pair after your analysis.

 

[Miner]

You can run another experiment with the two most likely candidates and test those separately.