Why not go with tests dealing with unequal variances/unbalanced sample size first?

[royk4444]

Hi All,
Here is the context: a lot of texts for hypothesis testing/ANOVA first examine
the cases where either the number of samples per group are the same
(n1 = n2 = n3 = ...= nn) or the variances for each group are comparable
(s1 = s2 = s3 =... = sn). And a lot of times, I see "Conduct the test and then
check to see if the variance assumption is violated." If it is, then do something
else... My question is why not just go with the more robust test (nonbalanced/
nonequal variance)from the start. Wouldn't the tests naturally converge to the
same results of the standard test (balanced/equal variance)? Are there cases where
this does not happen?

RK

 

[trinker]

Re: Why not go with tests dealing with unequal variances/unbalanced sample size first

They may lack the power to reject though the default of R's t-test is to assume unequal variance.

 

[spunky]

Re: Why not go with tests dealing with unequal variances/unbalanced sample size first

My question is why not just go with the more robust test (nonbalanced/
nonequal variance)from the start. Wouldn't the tests naturally converge to the
same results of the standard test (balanced/equal variance)? Are there cases where
this does not happen?

it is also true that in many cases no robust alternative exists to the traditional test, or the robust alternatives are so complicated that don't even warrant the effort.

there're quite a bit of times where you're better off using a traditional, parametric test with some sort of correction due to violation of assumptions rather than going the robust way.

keep in mind that it's not unusual for data to violate the assumptions of a test (i'd say that's probably the standard). it's how much you can get away with violating the assumptions and still be able to make valid inferences from your data.

 

[CB]

Re: Why not go with tests dealing with unequal variances/unbalanced sample size first

I think this is a good question. Say you have a situation where:

1) there is a robust alternative that doesn't add undue complications to the running of the analysis or interpretation of the results
2) the robust test is known to perform well when the assumption of concern is violated
3) the robust test is known not to do substantially worse than the conventional parametric test when the assumption is actually met.

Then it's pretty reasonable to just pick the robust test every time. A good example is a t-test (here is an article suggesting we just always use the Welch's version, without testing for equality of variances).

But a lot of the time, conditions 1-3 may not be met. E.g.,

1) The robust test may be hard to implement, or change the hypothesis being tested (e.g., rank-based tests such as the Mann-Whitney seem similar to their parametric equivalents, but test quite different null hypotheses)
2) The supposedly robust test may actually not be (e.g., the "asymptotically distribution free" estimator in SEM performs poorly in comparison to more conventional estimators unless you have a massive sample size)
3) The robust alternative may have less power than the parametric test when its assumptions are met.

 

[Englund]

Re: Why not go with tests dealing with unequal variances/unbalanced sample size first

here is an article suggesting we just always use the Welch's version, without testing for equality of variances

Wow, I am so glad to read that, because I came to the exact same conclusion after conducting some simulations using the Aspin-Welch test. I reported this in front of class and one of the professors said "Hrrrrfff, I think you have to think twice about this....hmmrmrmrs".