Order Effects- Conflicting results from two-way ANOVA and split file t tests!


Whilst currently working on my dissertation I decided to check for order effects (I had of course used counterbalancing as part of the design). My experiment looked at the effect of motivational prime on categorisation and probabilistic reasoning. There were two groups and both groups had to complete two tasks (one related to each dependent variable). On one of the tasks (categorisation) I found a significant difference in score between group A and Group B.

Initially, I split file by Order (order referring to which test was presented first) and proceeded to conduct an independent t test comparing groups, which revealed that this significant difference was only present in Order 1 and not Order 2 (mean difference for order 1 =2.09 and mean difference for order 2=2). Groups sample size-11/experimental group at split file (originally 22/group).

Then I thought I should have really done a two-way ANOVA first, so I conducted one with order and group as fixed factors and categorisation score as dependent factor. The SPSS output revealed no significant interaction between order and group suggesting (if I understand correctly) that the effects of group on dependent variable is not different for order 1 compared to order 2.

My question is how can this be? Is either of these analyses inappropriate for the statistical question, if so how? Have I interpreted this incorrectly? Or is it a case of power? I am not sure what to report...

Any clarification would be much appreciated! and I apologise if any of what I have written seems unnecessary, I am not sure what information you might need to answer my question. Of course if you need further information, please let me know!

Thank you for taking the time to read this post!