Closed procedure for multiple comparisons

dmk

New Member
#1
I'm trying to determine if the closed procedure can be used in a certain clinical trial that includes several hypothesis tests, so I pulled the paper all others cite (Marcus et al, Biometrika (1976), 63, 3, 655-60) and I'm trying to understand it. However, I'm having trouble already in the first paragraph:

Let X be a random variable with distribution Pθ(θ ϵ Ω). Let W = {ωβ} be a set of null hypotheses, i.e., a set of subsets of Ω (Question#1: how can each null hypothesis be a subset of Ω, which seems to me to be a set of parameters of the distributions of X?), closed under intersection: ωi, ωj ϵ W implies ωi ∩ ωj ϵ W. For each ωβ let φβ(X) be a level α test. That is, prθβ(X)=1} ≤ α for all θ ϵ ωβ (Question#2: Why is the test=1? I understood φβ(X) to be the test statistics, so why would the probability of it being 1 be less than α? Why 1, specifically?)

I'd appreciate if anyone can guide me to the answers!
 

hlsmith

Omega Contributor
#2
I looked at it last night and the "=1" part was not clear to me either. At first I though it may be related to some type of presumption about all tests being null, but got jammed up. Perhaps there is another paper that better presents the concept.
 

Dason

Ambassador to the humans
#3
Q1) They're saying that the set of parameters that the null hypothesis represents is a subset of the parameter space.
Q2) I'm guessing φβ(X)=1 indicates that the test rejected the null hypothesis and φβ(X)=0 implies failure to reject. We typically have alpha be the maximum type-I error rate. Some tests are conservative so that even if you use alpha = .05 as your criteria the true alpha is < .05. So they're just defining what it means for a test to be of level alpha.
 

j58

Active Member
#4
@dmk, the Marcus paper is a pretty read. You might find the book Multiple Comparisons in R by Bretz, Hothorn, and Westfal, which you should be able to find free online, a helpful companion.
 

j58

Active Member
#5
After reading about this a bit, the procedure seems very general. I can't think of any scenario in which it could not be, in principle, applied. Although how to implement it in a clinical trial if you're doing interim analyses might be complicated.
 

dmk

New Member
#6
@dmk, the Marcus paper is a pretty read. You might find the book Multiple Comparisons in R by Bretz, Hothorn, and Westfal, which you should be able to find free online, a helpful companion.
Thanks J58 for the book suggestion! The closed procedure is much better explained in it than in the Marcus paper.