2-way ANOVA Normality Requirements

[Ben Packard]

Greetings,

My objective is to conduct a 2-way anova (equal sample sizes/246 samples) in R. After running the Shapiro-Wilk normality test on the residuals, I am getting a p value of =.0011 which signifies that the distributions are not normally distributed. However, the histogram of errors and the Q-Q plot(attached) indicate that the nonnormality is at both extremes. A colleague looked at the plots and his recommendation was that I use us an ANOVA rather a non-parametric test as long as I included the Q-Q plot and/or histogram to support the decision. Therefore, I am wondering if it is acceptable to run a 2-way anova with this data. If it is not acceptable, what non-parametric test can I use (using R) in place of the 2-way anova?

Thanks, very much,

Ben

 

[Dason]

Were you running the normality test on the raw data or on the residuals from the analysis? The assumption of normality is on the error term so it doesn't make sense to do a test of normality on the data itself.

 

[hlsmith]

Those are some good looking graphs. I believe the different normality tests have "optimal" sample size ranges. You may want to look those up and see if the Shapiro-Wilk test appears to be best for your data.

 

[noetsi]

I note in passing that not all statisicians agree that normality of the raw data does not matter (I posted an example from two well known ones a while back on this topic). In areas like six sigma you will be taught to be sure the raw data is normal (as I was constantly taught throughout ANOVA....)

 

[Dason]

YOU WERE TAUGHT WRONG. Unless you're talking about the raw data within each group.

Seriously - I can literally derive the results and I could show you why we need the assumptions that we make. The assumption of normality is on the error term - not the raw data. We can make the assumption that conditioned on the group we're in the raw data is normally distributed - but it's much easier to say that the error term is normally distributed.

 

[Ben Packard]

Hi Everyone,

Thanks, very much, for your help and input. I apologize, but in the post I didn't explain that the attached plot and histogram are of the residuals from the analysis, not the raw data. The reason I thought I could still run an Anova was due to the non-normality being at both extremes as seen on the Q-Q plot.

 

[hlsmith]

I can't remember the cut-offs right now, but I believe a basic one is < 2000 S-W and > 2000 K-S, so given the approximate number of errors in your attached histogram, that may be the best test. But graphically they look pretty normal.

Dason, examining normality for GLM models is pretty straightforward by looking at the errors, but what do you look at when examining normality for two-sample t-test?

 

[noetsi]

YOU WERE TAUGHT WRONG. Unless you're talking about the raw data within each group.

Seriously - I can literally derive the results and I could show you why we need the assumptions that we make. The assumption of normality is on the error term - not the raw data. We can make the assumption that conditioned on the group we're in the raw data is normally distributed - but it's much easier to say that the error term is normally distributed.

You need to go back and reread the comments I cited from two very well known statisticians that argued very strongly that the raw data needs to be normal To a point they did not raise directly, t tests dont even have residuals, yet their data must be normal. Also my statistics professor strongly disagreed with you on this issue as well

 

[Dason]

t-tests do have residuals though. The t-test is a special case of anova which is just a special case of a linear model. The residuals are just the observations minus the corresponding group means.

@noetsi - I remember you posting some stuff that I did not find convincing. Given what I know and can prove... I'm sorry I just don't buy what you're selling.

 

[noetsi]

It really is not what I am selling. Your knowledge of statistics is a billion times mine any day. But the authors I cited, who are well known statisticians given their position, and wrote statistical text seemed pretty conclusive to me. I am not discussing the psychologists I originally posted - I meant the statisticians at Mount Sinai I posted later.

 

[Dason]

I still don't buy it. Fisher himself could come up to me and punch me in the face and tell me I'm wrong on this issue. I would be forced to buy him a beer and listen to what he has to say but that doesn't mean I think he's right.

If you're well known then you get a certain level of respect - but that doesn't mean that you're right. I also don't remember exactly which post or reference you're referring to so it's possible that you're just misinterpreting what they said or that they didn't do a good job expressing what they meant.

It's possible that they were saying the response needs to be multivariate normally distributed given the covariates. I'll buy that.

 

[GretaGarbo]

examining normality for GLM models is pretty straightforward by looking at the errors, but what do you look at when examining normality for two-sample t-test?

Since a t-test is just an analysis of variance (aov) with one factor on two levels, the examination of normality is done with the residuals as usual.

Noetsi wrote:

reread the comments I cited from two very well known statisticians 
that argued very strongly that the raw data needs to be normal

It would be nice if Noetsi showed the link to these statements.

(Dason and Noetsi commented while I was typing.)

 

[Ben Packard]

I can't remember the cut-offs right now, but I believe a basic one is < 2000 S-W and > 2000 K-S, so given the approximate number of errors in your attached histogram, that may be the best test. But graphically they look pretty normal.

Dason, examining normality for GLM models is pretty straightforward by looking at the errors, but what do you look at when examining normality for two-sample t-test?

Thanks,

The sample size was 246. I also ran the KS test in R to compare the anova residuals to a normal distribution. I used the following to do the test:

ks.test(errors, pnorm)

The result was p-value of .000707, which, if I'm not mistaken, indicates nonnormality.

The problem is that the histogram and Q-Q plot look like they are close enough to normal that I could use an ANOVA.

 

[Dason]

Normality isn't a huge issue. Your graphics look close enough to normality that we aren't worried too much (they really aren't perfect and the qqplot could be nicer but they're nothing to write home about). The sample size is large enough that the central limit theorem will fix any problems we had with non-normality.

Don't worry - you're fine.

 

[Ben Packard]

Thanks Dason,

Would you recommend that I include the Q-Q plot, the histogram, or both in the manuscript containing the ANOVA results?

 

[Dason]

I guess it depends on where you're submitting it to? Can you check similar articles to yours and see what they provide?

 

[noetsi]

To me a q-q plot is much better than a histogram at showing normality, but Dason is exactly right that you should consider the norm for your audience when chosing what information to provide. In my department test of skewness and kurtosis are actually favored over either of these in discussing normality. It varies.

 

[GretaGarbo]

Since you say you run it in R you cold try it in gamlss package with a box-cox-t distribution. (No, it does not have to be normally distributed, not even the residuals. There are many other distributions.)

The t-distribution has heavy tails, like in your histogram. If the degrees of freedom are large it will of course be approximately normal. The degrees of freedom can be estimated. If it is a t-distribution with just one degrees of freedom it will be a Cauchy distribution.

(The Cauchy distribution has been mentioned in quite confusing circumstances, here on this forum but let us ignore that.)

gamlss stands for “generalized additive models, location scale and shape”. Even if the model can not estimate the mean in the Cauchy distribution, it can estimate the location and the regression model that affects the location.

It seems from your histogram that there is some skewness. The box-cox part can take care of that.

Se www.gamlss.org

If you have space for it is nice if you include the histogram and (on second priority) the quantile-quantile plot.

 

[Ben Packard]

Thanks Greta,

Do you think using the gamlss package would be worth the time and effort as compared to just running the Anova. I'm fairly new to using R, and just have a basic understanding of statistics from taking a biostatistics class so it would take me a while to figure it out.

 

[hlsmith]

Depends on the journal and area of research. In medicine we hardly ever present normality graphs, you just take the author's word on it. Typically I would write normality assumptions were tested using #### and graphically reviewed (something like this). Then never present the figures because the space can be better used for other tables and figures about the actual study question. If the article was on statistical procedures or your computations then it might be applicable, but otherwise I would draft a sentence in the methods and if a reviewer asks, you can provide the figures during the review process.

If the graphs present some interesting information about the variables, you could always upload them as supplemental figures available online, but not in the publication.