Robust standard errors and REML

#1
Hello everyone,

Might anyone know why, when running multilevel (mixed effects) models with REML I get completely different results when I run the same models with robust variances? I originally chose REML because I have a small number of clusters (11). However, when plotting the residuals for a particular model, it was clear that they were nowhere near normally distributed. I re-ran the model using robust standard errors (the robust option is not available with REML in stata), and the results were completely different. Can anyone explain why this might be?

TIA.
 

spunky

Doesn't actually exist
#4
Both - variables are now significant that weren't before.
This is to be expected because both approaches are doing different things to the data. But different regression coefficients? Nah, that seems buggy.

Although...wait. When you said "I re-ran the model using robust standard errors" does that mean you ran OLS regression with robust/sandwich standard errors?
 
#5
This is to be expected because both approaches are doing different things to the data. But different regression coefficients? Nah, that seems buggy.

Although...wait. When you said "I re-ran the model using robust standard errors" does that mean you ran OLS regression with robust/sandwich standard errors?
Hello there,

I didn't explain properly, my apologies: I'm running a multilevel/mixed effects model and the first time I ran the model was with REML. I then plotted the predicted residuals and saw they were nowhere near normally distributed. I know that REML is fairly robust to deviations from normality, but on reading further current advice seems to be to use linear regression with robust errors. In stata, you can't use robust SE's with REML (I think you can in some other programmes), but when I re-ran the model using mixed effects linear regression with robust SE's the coefficients have indeed changed but not a huge amount, but the pvalues are veeeeerrrrryy different.
 

spunky

Doesn't actually exist
#6
Ok, this makes more sense now.

Sure, the p-values should change because you're messing the the standard errors in one analysis (using the robust options) and in the other you're letting them be what they are. And you have a small number of clusters (11) which implies a small sample at the cluster level. Small sample = more variability in the results.

So.... I dunno. In my mind, the scenario that you describe makes perfect sense why the inferences should be different but the coefficients should be (roughly) the same. I was a little concerned about the coefficients themselves changing because that shouldn't happen but now that you said they didn't change much then I think you're on the clear here.