Interpreting significance based on confidence interval

#1
Hi Everyone,
Could some of you help to interpret my results based on p values and confidence intervals? A little bit of background: I am trying to analyse the function of a gene during bacterial infection. So I have cell that express this gene, called naive and cell that do not express my gene, called mutant. Then I infect those cells with bacteria and after 24 hours I count the bacteria inside the cell to see whether the expression of this gene affects the bacteria growth. In this case, I ran the Mann-Whitnney test to detect statistical significance between the two groups and what I found was (p value: 0,0008; 95% CI 0,1040 to 0,3490). In term of statistics this is very significant. However, analyzing the data you do see that this difference is strictly small and insignificant. Here my question is: how the confidence interval could help me with this interpretation? In another experiment (called EXP IIa I got the following values (p value: < 0,0001; 95%CI 4,440 to 4,646). In this case, I do believe that significance is true because I have a drop of 5 logs on bacterial growth in the mutant group.
Could anyone help me with those interpretations?
Thank you
Best
Julian
 

hlsmith

Not a robit
#2
Wilcoxon-Mann-Whitney is considered the non-parametric analog of a t-test. It is typically used when there are normality concerns and skewed data. I am guessing this is why you used it.


Also can you provide examples of the bacterial growth data. You say counts, but I would be interested in how you are formatting it. Also, what is your sample size.


Lastly, what does your 95% CI represent, what value is it for. If it is for your growth variable that may be skewed, it may not be a good representation if your distributions are asymmetrical.
 

gianmarco

TS Contributor
#3
Like hlsmith, I do not understand what the CI is actually referring to. It would also be helpful if you can tell us which software you are using.

Mann-Whitney has indeed its "own" effect-size measures, like the Probability of Superiority and another one named "r". You could try and search on the web for those two.

Further, you did not tell us about your sample size. It could be that you have a very large sample size, and therefore even a small (practically negligible) difference proves statistically significant. But, not knowing the details of your analysis and data, we (well, I) cannot tell you more.

Best
Gm
 
#4
Hi,

Thanks gianmarco and hlsmith for your comments.
The software I use is Graphpad prism version 6. My sample size is 9 (three indenpendent experiments and 3 replicates each). When I say counts, I refers the yields of bacterial DNA quantified by qRT-PCR.
Below the raw data of my statistical analysis.
Thank you

Best
Experiment: Bacteria Internalization [/B]
Table Analyzed STAT S aureus

Column A MEF-wt
vs. vs,
Column B MEF-pprg-/-

Mann Whitney test
P value 0,0008
Exact or approximate P value? Exact
P value summary ***
Significantly different? (P < 0.05) Yes
One- or two-tailed P value? Two-tailed
Sum of ranks in column A,B 121,0 , 50,00
Mann-Whitney U 5,000

Difference between medians
Median of column A 6,176
Median of column B 5,964
Difference: Actual 0,2125
Difference: Hodges-Lehmann 0,2266
96.01% CI of difference 0,1040 to 0,3490
Exact or approximate CI? Exact

MEF-wt MEF-pprg-/-
Number of values 9 9

Minimum 5,964 5,737
25% Percentile 6,090 5,835
Median 6,176 5,964
75% Percentile 6,246 6,035
Maximum 6,313 6,086

Mean 6,158 5,933
Std. Deviation 0,1092 0,1169
Std. Error of Mean 0,03639 0,03898

Lower 95% CI 6,074 5,843
Upper 95% CI 6,242 6,023

Mean ranks 13,44 5,556


Experiment Bacterial Growth

Table Analyzed STAT S aureus

Column A MEF-wt
vs. vs,
Column B MEF-pprg-/-

Mann Whitney test
P value < 0,0001
Exact or approximate P value? Exact
P value summary ****
Significantly different? (P < 0.05) Yes
One- or two-tailed P value? Two-tailed
Sum of ranks in column A,B 126,0 , 45,00
Mann-Whitney U 0,0

Difference between medians
Median of column A 7,181
Median of column B 4,091
Difference: Actual 3,091
Difference: Hodges-Lehmann 3,097
96.01% CI of difference 3,048 to 3,180
Exact or approximate CI? Exact

MEF-wt MEF-pprg-/-
Number of values 9 9

Minimum 7,121 3,751
25% Percentile 7,153 3,998
Median 7,181 4,091
75% Percentile 7,213 4,124
Maximum 7,234 4,142

Mean 7,184 4,047
Std. Deviation 0,03793 0,1259
Std. Error of Mean 0,01264 0,04196

Lower 95% CI 7,155 3,950
Upper 95% CI 7,213 4,143

Mean ranks 14,00 5,000