Estimated Means in GLMM put a group's score as sig higher, but it should be sig lower

#1
Hi!
I ran an experiment where participants took a test of 16 questions, yes/no (binary).
They were tested at either 7-days or 28-days - two groups between subjects.
Analayzed as subjects, their results for the test are higher at RI-7.
RI-7: 5.5
RI-28: 3.6
t-tests confirm that this difference is sig - t = 2.171, p = .032
Analyzed as items, RI-7 scores are still higher.
RI-7: 515 incorrect - 269 correct
RI-28: 596 incorrect - 172 correct (more incorrect, fewer incorrect).
However
When put into a GLMM (binomial, logit), RI-7 scores come out sig lower, no matter what I do and what other variables I put in or take out.
An example from one model:
RI effect: F = 28.542 p = <.001
RI-7->RI-28 coefficient -.625, OR .535, p = <.001
EMMs: RI-7: .668; RI-28: .790
(more detailed stats in attachment)
Can anyone explain how this is possible or give ideas for something I can check?
Thanks in advance!!
 

Attachments

Last edited:

hlsmith

Less is more. Stay pure. Stay poor.
#4
What does the outcome look like format-wise? Just trying to figure out how result goes from continuous to binary?
 
#5
The first descriptives I gave are from an aggregated file.
The cross tabs and GLMM are from a by-items file.
e.g.
ID Item Score
1 -- 1 -- 0
1 -- 2 -- 1
1 -- 3 -- 0
1 -- 4 -- 1
1 -- 5 -- 1
1 -- 6 -- 1
1 -- 7 -- 1
1 -- 8 -- 1
1 -- 9 -- 0
1 -- 10 -- 0
1 -- 11 -- 1
1 -- 12 -- 1
1 -- 13 -- 0
1 -- 14 -- 0
1 -- 15 -- 1
1 -- 16 -- 1
 

hlsmith

Less is more. Stay pure. Stay poor.
#6
I was just trying to probe and convey that they may not align, since they are two different tests, right?