Power analysis for mixed-effects model

[LucieR]

Does anyone know how I can run a power analysis to determine the sample size for a linear mixed-effects model? Can this be done on G*Power?

Thanks

 

[hlsmith]

@spunky - what are you doing? Answer this question NOW.

 

[Dason]

I not sure about g power. But you can always simulate to estimate power.

 

[spunky]

Does anyone know how I can run a power analysis to determine the sample size for a linear mixed-effects model? Can this be done on G*Power?

Thanks

It can't be done in GPower. You'll need to approximate it through computer simulations,

 

[Buckeye]

Maybe I read or heard somewhere that power analysis' are often a simplification of the model that generates the data. As you start getting in the weeds with more complicated models, a power analysis for that specific model is harder to calculate. So, you do your best with what's tractable.

 

[Dason]

Maybe I read or heard somewhere that power analysis' are often a simplification of the model that generates the data. As you start getting in the weeds with more complicated models, a power analysis for that specific model is harder to calculate. So, you do your best with what's tractable.

Or you just simulate

 

[LucieR]

How would I simulate it? I'm guessing this is something that needs to be done on R?

 

[Dason]

It could be done in any reasonable programming language. R would be the easiest though. Basically you just need to fully write out what the data generation process looks like under your proposed alternative hypothesis and then use built in random number generators to simulate the values from those distributions and then do the analysis. One nice benefit of this approach is that the code for analyzing the data is already created after this point.

If you have specific questions we'd be more than happy to help but without specific details that's about all I can say.

 

[LucieR]

Unfortunately, I don't really have any experience in R/programming in general (beyond the basics). I ran the mixed-effects models in Jamovi - they included one continuous predictor, one categorical predictor, and two cluster variables (two random intercepts). I don't suppose there is any other way this can be done?

 

[Dason]

I believe jamovi provides a mode to get the corresponding R code for any analysis. That would be a good place to start. From there it's more about generating the code for the data generating process.

 

[LucieR]

Thank you, I will have a look into that!

Could this be an alternative option - https://jakewestfall.shinyapps.io/two_factor_power/? The only issue is that you need the variance estimates to calculate the VPC which I can only get post-hoc. Is there any way around this so that I can determine the appropriate sample size using this app beforehand?

 

[Dason]

Could this be an alternative option - https://jakewestfall.shinyapps.io/two_factor_power/? The only issue is that you need the variance estimates to calculate the VPC which I can only get post-hoc. Is there any way around this so that I can determine the appropriate sample size using this app beforehand?

I'm too lazy to dig into the details but it just so happens I think I know just the person that might be able to provide input on that specific question.

@Jake think you could help here?

 

[spunky]

I thought @Jake's app could only handle categorical predictors. As in it is meant to be used as a fairly sophisticated ANOVA.

 

[Jake]

What wretched soul has summoned me to the board?

Yes, all my mixed model power apps are for all categorical predictors only. Not only that, but they are for balanced, orthogonal designs. Those are the only special cases in which the math works out simply enough that you can get easy analytic power calculations. If you're dealing with continuous predictors and especially correlated continuous predictors, then I recommend simulation.

 

[Dason]

What wretched soul has summoned me to the board?

Yes, all my mixed model power apps are for all categorical predictors only. Not only that, but they are for balanced, orthogonal designs. Those are the only special cases in which the math works out simply enough that you can get easy analytic power calculations. If you're dealing with continuous predictors and especially correlated continuous predictors, then I recommend simulation.

That was my thought as well but since they literally referenced your shiny app I had to summon you.

 

[Dason]

With that said it might be interesting to see how different the results are between the full simulation compared to splitting the continuous predictor into a binary variable at least for the case of power or sample size estimation.

I'm literally just spitballling and do think pure simulation is the ideal approach though.

 

[spunky]

SIMULATIOOOOON!

Yeah, I just wanted to add my voice to the pro-simulation chorus to approximate power analysis.

 

[Jake]

With that said it might be interesting to see how different the results are between the full simulation compared to splitting the continuous predictor into a binary variable at least for the case of power or sample size estimation.

I've recommended that to a person or two in the past. Although the app linked in this thread deals with binary predictors, my PANGEA app supports categorical variables with up to like 12 levels or something, as well as arbitrary contrasts. So my thinking has been that if you specify a 12-level categorical predictor and then compute the power for a linear contrast, it's probably a pretty reasonable approximation to the continuous case. A long time ago I wanted to investigate this approximation more closely, but I never got around to it, and now I don't really care anymore.

That approximation strategy is just for the case of a single predictor. Like I hinted earlier, things are much more difficult with multiple predictors, especially if they are correlated, for reasons I won't list here but which I'm sure you can have fun imagining.

 

[LucieR]

Apologies for the late reply - I have been off the last few days with covid. Thank you for all the suggestions. It looks like simulation is the way to go!

I don't have any experience in this so if anyone has any resources that may help this would be very much appreciated.

 

[hlsmith]

I did this last week. Not the best approach, though it worked. Another approach is writing out your model with coefficients and simulating data for what you plan to collect. Feel free to start a new post and I would be happy to try and help - though I am fairly unseasoned doing them with MLM data, since you kind of have a model in a model.

(4) Power Calculation for a Multilevel Linear Regression | Statistics Help @ Talk Stats Forum