reporting statistical significance & P values

#1
Hi all, :wave:

I hope someone will be able to help me with this or point me in the right direction. I am writing up a report for Uni and have run an independent samples t-test on my data in SPSS. I get a p value of 0.049, less than 0.05 but only just.

Then I split my sample approximately in half (I wanted to separate people who spoke one language from people who spoke another). I have run the independent samples T test on each of the groups separately. Now they both come out as non significant.

I am not quite sure how to report this, can I say the results for the groups separately indicate I should question my 0.049 as indicative of a significant interaction and should use a more conservative critical value?

Why is it I get a significant result in one instance and then halving the sample size yeilds non significant results.

Is the significance I am finding for the whole group happening because the larger sample size is increasing type 1 errors?

Any tips, ideas, suggestions for something to read or look up, would be much appreciated!
 
#2
Hi all, :wave:

I hope someone will be able to help me with this or point me in the right direction. I am writing up a report for Uni and have run an independent samples t-test on my data in SPSS. I get a p value of 0.049, less than 0.05 but only just.

Then I split my sample approximately in half (I wanted to separate people who spoke one language from people who spoke another). I have run the independent samples T test on each of the groups separately. Now they both come out as non significant.

I am not quite sure how to report this, can I say the results for the groups separately indicate I should question my 0.049 as indicative of a significant interaction and should use a more conservative critical value?

Why is it I get a significant result in one instance and then halving the sample size yeilds non significant results.

Is the significance I am finding for the whole group happening because the larger sample size is increasing type 1 errors?

Any tips, ideas, suggestions for something to read or look up, would be much appreciated!

Sorry but i get i bit confused with your explanation. First i don't know what you are comparing?. Why you split your sample? and the question was the same in both analysis?. Why interaction? because you are running a t-test (only one factor with 2-levels), there not way to get interaction.

But i can tell you that if the condition for parametric test are ok (normality, but more important variance homocedasticity), then 0,049 is a significant value. You can choice to interpretate your results more conservatively, but any way you founded significative differences between groups.

Salu2s
 
#3
I am comparing the size of a memory bias testing familiar and unfamiliar words (some subjects are tested on familiar words, some unfamiliar words). I find word familiarity does influence the size of the memory bias p=0.049

Then I split my sample and test those whose first language is English and those whose isn't, separatley. When the two groups are tested seperately there is no significant interaction between word familiarity and the size of memory bias in either case.
 
#4
I am comparing the size of a memory bias testing familiar and unfamiliar words (some subjects are tested on familiar words, some unfamiliar words). I find word familiarity does influence the size of the memory bias p=0.049

Then I split my sample and test those whose first language is English and those whose isn't, separatley. When the two groups are tested seperately there is no significant interaction between word familiarity and the size of memory bias in either case.

Ok, now is better. Until i can see, you can perform two-way ANOVA, why? you have 2 factors with two levels each one: Factor 1= Words (2-levels: familiar and unfamiliar words); and factor 2= Language (2-levels: native and foreigner). Your models will look like this:

Y= constant + words + language +(word * language) + residuals

With this model you can explore separate and interacting effects between both factors. For example native speakers can memorise better familiar than unfamiliar word, and foreigner speakers vice versa. I hope that i understand good your problem???. I assume that you response variable is memory size, and you can tell me how you measure it?.

Salu2

Mauro

p.s. sorry for any english mistake, I'm a foreigner !!!:)
 
#5
No it's great you are really helping me, the problem is we have not yet got to studying ANOVA in my course!

So I can't really use better tools for my analysis, I just want to explain the output I have - finding significance in one case but not the other.

I am thinking that as I have split my sample, the interaction isn't big enough to show significance with the smaller sample? But it can when my sample is bigger?
 
#6
No it's great you are really helping me, the problem is we have not yet got to studying ANOVA in my course!

So I can't really use better tools for my analysis, I just want to explain the output I have - finding significance in one case but not the other.

I am thinking that as I have split my sample, the interaction isn't big enough to show significance with the smaller sample? But it can when my sample is bigger?

Hi i'm here again,

Ok, in part your thinking about of sample number could be right, but before to say anything you must to explore your data visually. With simple plots you will see what happen with the group-data distribution and try to take a decision.
I know that it's not really easy sometimes, but you are the people that know more about your experiment and here is coming the best part of the investigator game. Try to see what factor are affecting more the behaviour of your result. Again, if you think that the number of samples are important in your result then you can try to increase it or well give your interpretation being more conservative.

Ok i hope my comment help you, but also try to see another visions

Best wishes, Mauro