Mixed Repeated Measures ANOVA - some questions

#1
I have run an experiment and am analysing the results using a mixed RM ANOVA in SPSS. I have two queries, which I hope you can help me with. I apologise that this is a fairly long post -- I've tried to be as clear as possible, to help you to help me! -- and many thanks in advance for taking the time to read it.

First, here's the background to the experiment. It tested participants' judgements of the grammaticality of sentences. There were four different types of sentences tested (variable = "type", levels = 4), each of which appeared in an affirmative and negative form (variable = "polarity", levels = 2). There were two versions of the experiment (to ensure no ordering effects) and participants were assigned to one of four different age groups. So, the ANOVA has 2 within subjects factors: type and polarity; and 2 between subjects factors: age group and version.

My questions are as follows.

First, I know that ANOVA, being a parametric test, assumes a normal distribution. I'm confused about what this means, though: do the data have to be normally distributed? This is problematic for my study, because the expectation was that any participant who behaved like a normal adult should judge 3 out of 4 of the sentence types as grammatical. So, if I draw a histogram with percentage of grammatical responses on the X axis and frequency on the Y axis, then I expect a trend line to be shifted towards the right of the histogram (i.e. towards 100% judgements as grammatical). Does this mean that my data violates the normal distribution assumption, or am I thinking about this in the wrong way?

Second, when I ran the ANOVA, Levene's Test gives significant results, suggesting that the Homogeneity of Variance assumption has been violated. I'm not entirely sure I understand exactly what Levene's Test measures: am I correct in thinking that it's comparing the variance in the way participants respond for each sentence type*polarity? If that's correct, then it seems to me that I'd expect it to be significant, if the participants are more sure in their judgement of some sentence types than others. In any case, the most important questions here are should I worry about the fact that Levene's Test is significant, and, if so, what should I do about it?

Many thanks, if you've taken the time to read this far. I'd be really grateful if anyone can help me out with this.

Gary
 
#2
34 views and no comments. If it's my description of this that is unclear, please post and let me know -- I really need to get to the bottom of this ASAP! :)

Thanks,

Gary
 

bugman

Super Moderator
#3
Hi Gary,

this might be too simplistic for you needs, but I'll give it a crack.

Levenes test is a diagnostic test testing Ho: that k number of samples have equal variances. It is analysing the spread of your dependent variable and as far as I know is not considering the interaction term. You cannot makes inferences from your data based on Levenes test. It is used in a similar way that the Shapiro-Wilk test is a test from normality - that is, if your data is appropriate for parametric analysis. Normally distributed data will have a bell shape to it when plotted as a histogram.

One rule of thumb is that generally ANOVA is more robust to departures of normality than it is from having unequal variances. At the end of the day these tests are guides and your own judgement and common sense plays a big part (as long as you can justify it).

What are the p-values of you levenes test? Have you tried plotting your data to look at the distribution patterns?

Transforming your data might also help - in your case (your response variable is a proportion if I read your post correctly) arsine transformation are commonly used in my field for proportional data. Otherwise try a log transformation.

I hope this helps Garry.

Cheers,

Phil
 
#4
Thanks for this, Phil.

Levenes test is a diagnostic test testing Ho: that k number of samples have equal variances. It is analysing the spread of your dependent variable...
My DV here is the percentage of correct judgements. I would expect participants who are native speakers of English to consitently make correct judgements of grammaticality, and hence their scores should be closer to 100% than 0%. Young children, however, who have not fully learnt the language may not be so consistent in their judgements, so their distribution of % correct judgements will be different. Doesn't this mean that I'd expect there to be unequal variance?

...It is used in a similar way that the Shapiro-Wilk test is a test from normality - that is, if your data is appropriate for parametric analysis. Normally distributed data will have a bell shape to it when plotted as a histogram.
This isn't necessarily the case for each of my age groups tested. For the youngest group, whose correct responses are around chance performance, the curve is bell shaped like a normal distribution. However, for the older age groups, they give more correct responses (just as I'd predict, and as explained above). Does that mean these populations cannot be considered normal and that they violate the conditions of ANOVA?

What are the p-values of you levenes test?
p < 0.0005

Have you tried plotting your data to look at the distribution patterns?
I'm not sure what you mean here - are you talking about a histogram?

Thanks again,

Gary
 

bugman

Super Moderator
#5
Garry,

with your two groups (native speakers and young people) you might expect their mean response to be different, but not neccesarily their varaince. The variance is just a measure of spread around the mean. ANOVA assumes these to be approximatly equal.

Yes, I was refering to histograms, but there are other quick ways (such as boxplots).

If you are still stuck and you want someone to look at it for you - post it in private meassges and I can see what I can do.

Phil
 
#6
Any news about Levene's and ANOVA on skewed distributions?

Hi, I am having the same problem about skewed distribution. The same case of adults and children that garywood84 has. I was wondering how did you manage to solve it? Is it OK to run ANOVA even when the distribution is skewed?

What happened with the Levene's test? If I have a significant Levene's Test (p < 0.05) I should not run ANOVA, right? But what can I conclude from it being significant? Can I reject or fail to reject Ho?

Thanks!!!
 
#7
Hi, I am having the same problem about skewed distribution. The same case of adults and children that garywood84 has. I was wondering how did you manage to solve it? Is it OK to run ANOVA even when the distribution is skewed?

What happened with the Levene's test? If I have a significant Levene's Test (p < 0.05) I should not run ANOVA, right? But what can I conclude from it being significant? Can I reject or fail to reject Ho?

Thanks!!!

Howell (5th ed., 2002, p. 340) says not to worry (much) about non-normality in ANOVA, citing Box (1953, 1954a, 1954b), Boneau (1960), Bradley (1964), and Grissom (2000). But it is not okay to run the ANOVA if you have significant violations of the assumption of equality of variances, as indicated by the Levene's test. By the way, you are looking at the Levene's test output for your factor and not for the intercept, right?
 
#8
It is okay to run ANOVA when the data are skewed, but that generally means a slight skew.

The assumption in ANOVA (and linear regression) is that the conditional distribution of Y|X follows a normal distribution, not Y, the DV.

Proportions as dependent problems also have truncation problems, since they are limited to being between 0 and 1. So your dependent variable, the proportion, is limited to that range, but predicted values based on the right hand side of the equation, can be outside that value.

At least according to one of my clients, whom I just helped do a repeated measures logistic regression for this exact reason, a number of journals, at least in linguistics, are starting to reject papers that use ANOVA for proportions.

See this paper by Florian Jaeger:http://www.sciencedirect.com/scienc...serid=10&md5=4f908745bab1ad0a849b30eccb8d4c43.

I've also heard of using tobit regression for DVs that are proportions.

Karen