which test for repeated measures within-between group change with small sample size?

[srl0704]

Hi all,

I have an intervention study with a very small sample size and am trying to decide the best way to analyze it. Any suggestions would be extremely helpful. Here is some more information:

Measurements on 5 different variables were taken at 3 time points: Baseline, Follow-up 1, Follow-up 2

Group 1 (n=14) completed one set of exercises, Group 2 (n=16) completed a different set of exercises, and Group 3 (n=8) completed no exercises.

Ideally, I would like to compare change in all of these groups across all three time points, but my sample size is so small and I know having so many groups, time points, and measurements makes things even more difficult.

Does anyone have suggestions?

Analyses I am considering are:

repeated-measures ANOVA for each variable separately or a repeated-measures MANOVA for all variables, multilevel modeling, separate repeated measures t-tests for each group on each measure (so no statistical between-group comparison), regression with baseline scores as covariates….help!! Thank you so much!

 

[mmercker]

Re: which test for repeated measures within-between group change with small sample si

Hi,

all the approaches you mention make sense and would have their legitimation. However, the bottleneck in your study is the small sample size, so I am not sure if you will obtain any significant results.

So maybe a possible start would be to analyze each outcome variable separately in a repeated-measures ANOVA / multilevel Model (which should both give the same results) using "Time point" and "Group" as categorical predictors, and the participant ID as the random factor. If you think that your outcome variables are correlated, MANOVA is a reasonable alternative.

 

[the42up]

Re: which test for repeated measures within-between group change with small sample si

Yes, small sample size is an issue but not something you cannot overcome. A repeated measure design is the safest bet.

Out of curiosity, do you have any information from people who dropped out of the study? Your groups appear unbalanced, is that because you are doing listwise deletion? If that is the case, I would probably not choose that method and employ a mixed effects model rather than a repeated measures design. You increase complexity but you might have the power you need to detect a statistically significant effect size.

 

[srl0704]

Re: which test for repeated measures within-between group change with small sample si

Yes, small sample size is an issue but not something you cannot overcome. A repeated measure design is the safest bet.

Out of curiosity, do you have any information from people who dropped out of the study? Your groups appear unbalanced, is that because you are doing listwise deletion? If that is the case, I would probably not choose that method and employ a mixed effects model rather than a repeated measures design. You increase complexity but you might have the power you need to detect a statistically significant effect size.

Hello, yes there were more people who started the study than completed each of the follow-ups, so those people were removed. Do you have a better recommendation for handling the missing data?? Thank you!

 

[the42up]

Re: which test for repeated measures within-between group change with small sample si

Here are my go-to approaches for mixed models-

A Wald T as your test statistic (Verbeke & Molenberghs, 2009)
A Satterthwaite approximation to calculate degrees of freedom for significance tests (Satterthwaite, 1946).
Ω squared, is a generalized form of R squared for mixed effects models (Xu, 2003).

So that is my approach to accommodate mixed effects models.

In your case, you will have a nested model with time points nested in subjects. This allows you to preserve the information from people who dropped out of the experiment.

It really depends on your comfort level with mixed effects linear models versus linear models. In this case, you can use a multi-level model- esque approach to preserve your information. You would have two levels- time points and subjects.