Non-parametric tests that do not assume equality of variance

[TacoX]

Hi all,
i have a Before-After-Control-Treatment (2x2) design, resulting in two independent groups of scale data (samples in two water streams, in one a treatment has been installed at some point, both streams (two physical different locations) were sampled before and after at various occasions). i would like to test for differences in distribution, the effect of this treatment.
My data are not-normal distributed, and some of the group combinations violate the assumption of equality of variance, as the output of the Modified Levene's test testified.
Initially I have performed Mann-Whitney U tests on the groups so far, with several outputs resulting a significant difference across the treatment.
But MW-U assumes equality of variance.
1) How sensitive is MW-U to the violation of equality of variances?
2) To determine an effect size of the treatment, for instance eta squared, using the Z statistic of MW-U, is equality of variance needed?
3) Are there any alternative post-hoc tests similar to the univariate Games-Howell test (which does not assume equality of variance), that can be run on non-parametric 2x2 designs?
ps: i am working in SPSS
Many Thanks,
Taco

 

[Karabiner]

My data are not-normal distributed,

That means, you have checked the distributions in all 4 cells separately?
Or what are you referring to with this statement?

and some of the group combinations violate the assumption of equality of variance, as the output of the Modified Levene's test testified.

How large is your total sample size?
Are group sizes very different? How large are the non-equalities of variance in the sample?

Initially I have performed Mann-Whitney U tests on the groups so far, with several outputs resulting a significant difference across the treatment.

What exactely did you test, using the Mann-Whitney U-test? Your design looks more like
something to be analysed by mixed analysis of variance (with a grouping factor and a
repeated-measures factor), for which the U-test is no alternative.

But MW-U assumes equality of variance.

Absolutely not so. The Mann-Whitney test is a test for ordinal data (ranked data).
Those data do not have variances.

With kind regards

Karabiner

 

[TacoX]

dear Karabiner,

thanks for the rapid response,
in the mean time i found a link that indeed confirms that MW-U can treat data that have differently shaped distributions.

i checked all possible group combinations and tested them for normality and homogeneity of variance. all outputs were significant at p < 0.05.
therefore, my data is not-normal distributed, and have unequal variances.
the design is as follows:


VarA is a storm response variable sampled in two natural streams, each in a different physical location in a catchment. storm responses variable may be e.g. discharge, peak discharge, or lag time. each value is a snapshot of the state of a stream, repeatedly collected in the same stream on different days, but not necessarily simultaneously in both streams.

one of the streams received a treatment half way in the experiment, and we measured the same variable over and over, but always in different conditions obviously (=weather).

with Mann-Whitney I tested for (when distributions were differently shaped):
1) the differences in distribution between stream responses per sample period
2) the differences in distribution between treatment periods per location.

i would like to determine an effect size for the treatment. and understand if the lack of homogeneity of variance influences the choice of effect size parameter?
i.e. can i use Cohen's d? or should i stick to eta squared?

thanks,

Taco

 

[Karabiner]

If you use the Mann-Whitney U-test, then you do not compare means and do not have
a test on interval data, so Cohen's d or eta squared are not adequate.

By the way, you cannot know an effect size, based on sample data. You can
calculate effect size measures, if you wish, but these are contaminated by
random error, so they do not represent the true effect size. That is why we
perform statistical inference, instead of just reporting effect size measures
derived from sample data.

I am not sure that I correctly understand your design and your dependent variables.
But if these are indeed independent groups, and interval scaled dependent
variables, then total sample size is > 30 in each group comparison, so that you
do not need normally distributed variables in each group, and you can use
the Welch-corrected t-test instead of the uncrrected t-test. It corrects for unequal
variances.

With kind regards

Karabiner

 

[TacoX]

i might go with that then. many thanks