When 'not' to use MANOVA?

[Vatsal]

Hey!

This is about my PhD thesis where I am trying to see the pattern of memory deficit among different types of hypothyroidism. I have used a between subject group design with 1 IV with four levels. Thus, I have 4 four groups which are:

Subclinical Hypothyroid group N= 14
Overt Hypothyroid group N= 15
Euthyroid group N = 12
Healthy Control Group N = 15

I am using multiple types of memory measures broadly speaking I have administered 1 test of verbal memory with 6 types of scores (DVs) and 2 Tests for working memory with 4 types of scores. I have been ask by some faculties y department to apply One-way MANOVA.

Coming to the issue I am facing is that applying One-way Manova would require meeting several assumptions which my data does not. My data fails to meet the following assumptions:

Normality (univariate)
Linearity within DVs
No of Cases per DVs

Moreover, I searched through my literature I did not found any previous author to use MANOVA to evaluate statistical difference between the groups. Most of the researcher employed pre-post test design but used univariate measures like ANOVA/t-test to measure the statistical difference. So in the light of my issues with my data and the previous researches should I go for multiple One-way ANOVA? What measures I should take to control type I error and also type II error???


Thanks

Vatsal

 

[staassis]

There is no minimum sample size requirement for MANOVA as long as one can calculate all the means and covariance matrices in each category. To make the data normal, you may want to experiment with various non-linear transformations (e.g. Box-Cox family).

Your professors are correct: some "aggregate" test like MANOVA is a good place to start. You have too many dependent variables given the overall sample size. Running a univariate test for each of them would result in multiple testing, and that would necessitate the Bonferroni adjustment. The Bonferroni adjustment is very conservative. Even moderately strong relationships would come out as statistically non-significant on a small data set like yours. Researchers try to avoid situations where the adjustment is required.

Think about this: even if there is absolutely nothing there, you would still be making a type I error 5% of the time.

 

[Vatsal]

Thanks for your reply staasis!

So I should go for MANOVA after transforming the data? Although I have tried squaroot and reflex squaroot but it messes up the variable even more. I have not yet tried box-cox transformation.

 

[staassis]

Yes, try

Box-Cox transformation with optimally chosen parameter -----> normality tests -----> MANOVA

MANOVA Wilks' Lambda test is not the only suitable "aggregate" test in this situation. However, it is a good starting point given your professors' preferences.

 

[Dason]

Keep in mind that if you're looking at the raw data that's not what we require normality on. You want to test the residuals after you run the analysis.

 

[Vatsal]

Yes, try

Box-Cox transformation with optimally chosen parameter -----> normality tests -----> MANOVA

MANOVA Wilks' Lambda test is not the only suitable "aggregate" test in this situation. However, it is a good starting point given your professors' preferences.

Under what circumstances Pillai trace should be used??

 

[Vatsal]

I

Keep in mind that if you're looking at the raw data that's not what we require normality on. You want to test the residuals after you run the analysis.

So I need to calculate residuals for each variable then see if they're normal??? But even if they turn out to be normal I need moderate correlation between DVs which is essential for MANOVA

 

[staassis]

Regarding the residuals, yes... A loosely equivalent way of testing whether the residuals are normal is testing whether the dependent variables are normal in each category (assuming there are no continuous predictors in your study)... There are no restrictions on the correlations between the dependent variables.

Most multivariate tests are comparable in the context of MANOVA. Almost always they produce very similar p-values. Wilks' is most accurate from the theoretical point of view.

 

[Vatsal]

Regarding the residuals, yes... A loosely equivalent way of testing whether the residuals are normal is testing whether the dependent variables are normal in each category (assuming there are no continuous predictors in your study)... There are no restrictions on the correlations between the dependent variables.

Most multivariate tests are comparable in the context of MANOVA. Almost always they produce very similar p-values. Wilks' is most accurate from the theoretical point of view.

Thanks. I didn't quite get your point on "continuous predictor" :/

 

[staassis]

Suppose you wanted to adjust for Age. That would be a continuous predictor (taking continuum of possible values).

 

[Vatsal]

Suppose you wanted to adjust for Age. That would be a continuous predictor (taking continuum of possible values).

Ok. There's one variables that is normal in only two groups while there is one thats not normal for the entire four groups.

 

[staassis]

As we said, use non-linear transformations.

 

[Karabiner]

I don't know how much sense a Box-Cox-transformed memory measure does make, or how interpretable it would be.
There seems to be no substantial reason for changing the data. Moreover, the recommendations seem to result
in a mixture of original data and transformed data. But anyway - is normality of the residuals essential here at all?
n(total)=56.

With kind regards

Karabiner

 

[staassis]

the recommendations seem to result in a mixture of original data and transformed data

Karabiner, what I meant was that the same non-linear transformation should be applied to all the categories. A Box-Cox transformation will not necessarily be the best one but the Box-Cox family is a good place to start. In particular, there are some built-in functions in R for the optimal parameter search.

The overall sample size is all right but in each category we have as little as 12 observations. So we do need normality for the regular MANOVA to be valid... Having that said, there is no guarantee that one "savior" transformation exists.