R and Stats: Multiple t-testing

[SteadyGrow99]

So 60 labs decided to group up to test the null hypothesis (sample mean is equal to zero) against the alternative (sample mean is not). The 60 labs collect 10 samples each and all use an α = 0.05 level. Assuming the null hypothesis is true:
a) Probability that no labs reject the null?
b) Probability that all labs reject the null?
c) Probability that exactly 10 labs reject the null?

Because it will be 60 t-tests, this leads me to think that false discovery rate or family-wise error rate may be applicable here (not certain). I'm not sure how to obtain the first and third probability, but I believe the second can be obtain by 1 - answer to a. Any help would be hugely appreciated!

 

[Dason]

I think it would only be 60 tests.

 

[SteadyGrow99]

I think it would only be 60 tests.

Thanks for clarifying that! Yes, 60 hypothesis tests on the 600 samples total

 

[Dason]

So if the null hypothesis is true (which the question says to assume) - what is the probability of rejecting the null?

 

[SteadyGrow99]

So if the null hypothesis is true (which the question says to assume) - what is the probability of rejecting the null?

We would reject the null if a p-value of less than 0.05 were obtained. I'm not sure if this answers your question; couldn't tell if you were asking about the significance level or wanted to clarify the topic questions.

 

[hlsmith]

alpha is the accepted rate of accepting a null results as being 'significant'. So you should expect 60*0.05 results to have a type I error. That is the jumping off point.

I have no idea if this is applicable at all, but I thought I recall that pvalues should be uniformly distributed.

 

[Dason]

I have no idea if this is applicable at all, but I thought I recall that pvalues should be uniformly distributed.

When the null is true then for the most part - yes. There are some technical requirements one needs to impose to get a perfectly uniform distribution but those requirements hold for this particular example. Really all that is needed though is that alpha is the probability of rejecting Ho when Ho is true.

 

[hlsmith]

@SteadyGrow99 - I am curious what you come-up with for answers. Please share when you make progress.

My brain hasn't solved this yet. I am thinking that I would define an underlying binomial distribution with a 5 percent chance and see what the probability of a 5 and 0 would be. I have meeting, so I know these are wrong but close to be defined.

Spoiler

pbinom(0, 60, 0.95)
pbinom(60, 60, 0.95)