Closed procedure for multiple comparisons

[dmk]

I'm trying to determine if the closed procedure can be used in a certain clinical trial that includes several hypothesis tests, so I pulled the paper all others cite (Marcus et al, Biometrika (1976), 63, 3, 655-60) and I'm trying to understand it. However, I'm having trouble already in the first paragraph:

Let X be a random variable with distribution Pθ(θ ϵ Ω). Let W = {ωβ} be a set of null hypotheses, i.e., a set of subsets of Ω (Question#1: how can each null hypothesis be a subset of Ω, which seems to me to be a set of parameters of the distributions of X?), closed under intersection: ωi, ωj ϵ W implies ωi ∩ ωj ϵ W. For each ωβ let φβ(X) be a level α test. That is, prθ{φβ(X)=1} ≤ α for all θ ϵ ωβ (Question#2: Why is the test=1? I understood φβ(X) to be the test statistics, so why would the probability of it being 1 be less than α? Why 1, specifically?)

I'd appreciate if anyone can guide me to the answers!

 

[hlsmith]

I looked at it last night and the "=1" part was not clear to me either. At first I though it may be related to some type of presumption about all tests being null, but got jammed up. Perhaps there is another paper that better presents the concept.

 

[Dason]

Q1) They're saying that the set of parameters that the null hypothesis represents is a subset of the parameter space.
Q2) I'm guessing φβ(X)=1 indicates that the test rejected the null hypothesis and φβ(X)=0 implies failure to reject. We typically have alpha be the maximum type-I error rate. Some tests are conservative so that even if you use alpha = .05 as your criteria the true alpha is < .05. So they're just defining what it means for a test to be of level alpha.

 

[j58]

@dmk, the Marcus paper is a pretty read. You might find the book Multiple Comparisons in R by Bretz, Hothorn, and Westfal, which you should be able to find free online, a helpful companion.

 

[j58]

After reading about this a bit, the procedure seems very general. I can't think of any scenario in which it could not be, in principle, applied. Although how to implement it in a clinical trial if you're doing interim analyses might be complicated.

 

[dmk]

@dmk, the Marcus paper is a pretty read. You might find the book Multiple Comparisons in R by Bretz, Hothorn, and Westfal, which you should be able to find free online, a helpful companion.

Thanks J58 for the book suggestion! The closed procedure is much better explained in it than in the Marcus paper.