Is a group unique or not?

#1
Seems like a simple question and I think I almost have the answer, but I'd appreciate a second opinion!

I have 4 groups of data ('a','b','c' and 'd' say). I want to know whether the mean of a is significantly different from *all* the others and have some kind of P-value giving the confidence that it is different from all the others.

So an ANOVA on the data can tell me whether one or more of the means differs from the others - but this could come up with a significant value if 'a' and 'b' have the same mean but differ from 'c' and 'd' for instance. I only want a significant value if 'a' is different from all the others.

So instead I can do 3 pairwise t-tests: a vs b, a vs c and a vs d. If all are significant then I know a has a unique mean. But now I have 3 P-values, each testing whether 'a' is different from one other group. Here's where I'm stuck: how do I boil those 3 P-values down to one which tells me the probability that 'a' is different from all the other groups.

What I'm thinking of doing is using

Pfinal = 1 - (1-Pab * 1-Pac * 1-Pad)

Where Pfinal is the P value I end up with (the probability that a is different from b and c and d), Pab is P value for the 'a' vs 'b' t-test, Pac is P value for the 'a' vs 'c' t-test and Pad is P value for the 'a' vs 'd t-test. Does that make sense?
 

JohnM

TS Contributor
#2
You can use Dunnet's t-test for comparison of a "control" or reference group (group a) versus several alternative groups.