Three replications in ANOVA

#1
Hi,
I have assessed many manuscripts submitted to a medical journal. I see that three replications are used in ANOVA commonly.
OK, I know that the analyses are expensive in terms of time and/or money. However, I am surprised and believe that the results are unreliable. The problem refers to the assumptions: normal data distribution in the compared groups and variance homogeneity. When n=3, the power of the test used in verifying those assumptions (Shapiro-Wilk's, Levene's etc.) is close to zero. To conclude, when the number of replications is very small, verifying the ANOVA'a assumption is extremely unreliable. Therefore, ANOVA should not be used when n=3, for example. However, even such journal as Nature publishes papers that present parametric ANOVA results based on n=3.

In my opinion, when reliable verifying parametric test assumption is impossible, the given test should be replaced with an appropriate nonparametric test (Kruskal-Wallis test, or PERMANOVA, etc. in the case of ANOVA).
Am I right or wrong?
 

Karabiner

TS Contributor
#4
To conclude, when the number of replications is very small, verifying the ANOVA'a assumption is extremely unreliable. Therefore, ANOVA should not be used when n=3, for example. However, even such journal as Nature publishes papers that present parametric ANOVA results based on n=3.
Maybe in those studies, based on external knowledge it is biologically plausible that some assumptions are fulfilled (e.g. normal distribution of errors)? Moreover, heterogeneity of variance is not considered harmful if the design is balanced (although i don't know whether this also applies for extremely small sample sizes in combination with large differences in variance).

One funny thing is, power is so low with such sample sizes (assuming that the tested true effects aren't huge),
that p < 0.05 is only reached if by chance the group differences in one's sample deviates largely from the true effect in the population.
And that means that any significant effect has a fair chance to point into the wrong direction
https://www.researchgate.net/figure...error-statistical-power-is-low_fig1_321895465

In my opinion, when reliable verifying parametric test assumption is impossible, the given test should be replaced with an appropriate nonparametric test (Kruskal-Wallis test, or PERMANOVA, etc. in the case of ANOVA).
I don't know whether K-W test can have a p < 0.05 with group sizes of n=3.

With kind regards

Karabiner
 
Last edited: