Power analysis for mixed-effects model

#1
Does anyone know how I can run a power analysis to determine the sample size for a linear mixed-effects model? Can this be done on G*Power?

Thanks
 

spunky

Can't make spagetti
#4
Does anyone know how I can run a power analysis to determine the sample size for a linear mixed-effects model? Can this be done on G*Power?

Thanks
It can't be done in GPower. You'll need to approximate it through computer simulations,
 

Buckeye

Active Member
#5
Maybe I read or heard somewhere that power analysis' are often a simplification of the model that generates the data. As you start getting in the weeds with more complicated models, a power analysis for that specific model is harder to calculate. So, you do your best with what's tractable.
 

Dason

Ambassador to the humans
#6
Maybe I read or heard somewhere that power analysis' are often a simplification of the model that generates the data. As you start getting in the weeds with more complicated models, a power analysis for that specific model is harder to calculate. So, you do your best with what's tractable.
Or you just simulate
 

Dason

Ambassador to the humans
#8
It could be done in any reasonable programming language. R would be the easiest though. Basically you just need to fully write out what the data generation process looks like under your proposed alternative hypothesis and then use built in random number generators to simulate the values from those distributions and then do the analysis. One nice benefit of this approach is that the code for analyzing the data is already created after this point.

If you have specific questions we'd be more than happy to help but without specific details that's about all I can say.
 
#9
Unfortunately, I don't really have any experience in R/programming in general (beyond the basics). I ran the mixed-effects models in Jamovi - they included one continuous predictor, one categorical predictor, and two cluster variables (two random intercepts). I don't suppose there is any other way this can be done?
 

Dason

Ambassador to the humans
#10
I believe jamovi provides a mode to get the corresponding R code for any analysis. That would be a good place to start. From there it's more about generating the code for the data generating process.
 

Dason

Ambassador to the humans
#12
Could this be an alternative option - https://jakewestfall.shinyapps.io/two_factor_power/? The only issue is that you need the variance estimates to calculate the VPC which I can only get post-hoc. Is there any way around this so that I can determine the appropriate sample size using this app beforehand?
I'm too lazy to dig into the details but it just so happens I think I know just the person that might be able to provide input on that specific question.

@Jake think you could help here?
 

Jake

Cookie Scientist
#14
What wretched soul has summoned me to the board?

Yes, all my mixed model power apps are for all categorical predictors only. Not only that, but they are for balanced, orthogonal designs. Those are the only special cases in which the math works out simply enough that you can get easy analytic power calculations. If you're dealing with continuous predictors and especially correlated continuous predictors, then I recommend simulation.
 

Dason

Ambassador to the humans
#15
What wretched soul has summoned me to the board?

Yes, all my mixed model power apps are for all categorical predictors only. Not only that, but they are for balanced, orthogonal designs. Those are the only special cases in which the math works out simply enough that you can get easy analytic power calculations. If you're dealing with continuous predictors and especially correlated continuous predictors, then I recommend simulation.
That was my thought as well but since they literally referenced your shiny app I had to summon you.
 

Dason

Ambassador to the humans
#16
With that said it might be interesting to see how different the results are between the full simulation compared to splitting the continuous predictor into a binary variable at least for the case of power or sample size estimation.

I'm literally just spitballling and do think pure simulation is the ideal approach though.
 

Jake

Cookie Scientist
#18
With that said it might be interesting to see how different the results are between the full simulation compared to splitting the continuous predictor into a binary variable at least for the case of power or sample size estimation.
I've recommended that to a person or two in the past. Although the app linked in this thread deals with binary predictors, my PANGEA app supports categorical variables with up to like 12 levels or something, as well as arbitrary contrasts. So my thinking has been that if you specify a 12-level categorical predictor and then compute the power for a linear contrast, it's probably a pretty reasonable approximation to the continuous case. A long time ago I wanted to investigate this approximation more closely, but I never got around to it, and now I don't really care anymore.

That approximation strategy is just for the case of a single predictor. Like I hinted earlier, things are much more difficult with multiple predictors, especially if they are correlated, for reasons I won't list here but which I'm sure you can have fun imagining.