I'm reading about the familywise error rate here and trying to put it into perspective. According to this, we calculate FWER = 1 - (1 - alpha)^n. In their example n=10, and with alpha .05, they say this gives 40.1% chance of a type 1 error. Is this practically saying that if i run a study and make 10 comparisons, the probability that an significant result (if one is observed!) is false is 40%? In other words, given that I see any significant, there is a 40% chance that this is false.? So what if I see 4 significant here? It's says approximately ~1.6 of those is false? I think i am right but want to be sure.
False discovery and multiple test
- Thread starter AlexanderC
- Start date
- Tags false discovery rate multiple comparisions
What it's saying is that even if all of the tests are such that the null hypothesis is actually true that there is a 40% that you will get at least one test coming out as significant. It doesn't say anything about the probability that a discovery is actually false or true or anything like that. Just that if all the tests are actually null we still have a 40% to get at least one significant result.