I'm tasked with taking over a job from someone else. It seems to me this person's been running an interim analysis monthly. Consider the following (extremely simplified) scenario: We have two groups: treatment and control.

Month 1: the person ran a two sample t-test to compare an average

Month 2: the person ran a two sample t-test with updated treatment and control groups

Month 3: the person ran a two sample t-test with updated treatment and control groups

Month 4: etc.

I know there are several issues with this setup:

1.) The response variable (whatever it is) will be correlated from one month to the next.

2.) Glossing over the fact that there are potential repeated measures from month to month

3.) I highly doubt the variance can be pooled.

4.) I don't know where to begin thinking about type I, type II, and power amongst other things.

5.) Issues I haven't thought of yet.

I've been doing some reading and I think this is similar to group sequential testing. I have not done this analysis before. But, I understand it is used in clinical studies when the treatment and control group change over time. Can anyone help me understand this? Ultimately, I think this boils down updating results as more data comes in.

Month 1: the person ran a two sample t-test to compare an average

Month 2: the person ran a two sample t-test with updated treatment and control groups

Month 3: the person ran a two sample t-test with updated treatment and control groups

Month 4: etc.

I know there are several issues with this setup:

1.) The response variable (whatever it is) will be correlated from one month to the next.

2.) Glossing over the fact that there are potential repeated measures from month to month

3.) I highly doubt the variance can be pooled.

4.) I don't know where to begin thinking about type I, type II, and power amongst other things.

5.) Issues I haven't thought of yet.

I've been doing some reading and I think this is similar to group sequential testing. I have not done this analysis before. But, I understand it is used in clinical studies when the treatment and control group change over time. Can anyone help me understand this? Ultimately, I think this boils down updating results as more data comes in.

Last edited: