Bayesian multiple comparison correction?

M

miramara

New Member
#1
#1
Hi everyone,

I am new to Bayesian statistics and am interested in the correlation of 4 "target" variables with 52 other variables. Currently I am running several separate Bayesian Spearman rank correlations. Many of the variables are highly correlated (up to .68). Finally, I would like to report the correlation coefficients (effects of >= 0.1 will be considered practically relevant) and their 95% CIs (no Bayes Factors so far; just estimation).

My (maybe not very smart) questions would be:
A) Do I have to apply multiple comparison correction? Is this necessary, given that I am not engaging in hypothesis testing but estimation, rather?

B) In case the answer to A is "Yes, you do need to correct for multiple comparisons." - is there a Bayesian way of doing this (ideally in R)?

In other questions related to this topic, the answer is usually that I would ideally just have one model in which I do not have to care about multiple comparison correction when comparing model parameters. However, this does not really match the situation I am in. I also see that I could engage in dimension reduction, but actually I would rather not (plus it wouldn't really help answering the question of whether, in principle, I need multiple comparison correction).

I would very much appreciate any answer! Thank you!!
 
hlsmith

hlsmith

Less is more. Stay pure. Stay poor.
#2
#2
This isn't a not very smart question, since it is not discussed often. You are doing well by just knowing that multiple pairwise comparisons can lead to false discovery.

I have never found a good description of what one should do. Frank Harrel (Vanderbilt) mentions this, but never fully clarifies. He usually states you don't have to do a post hoc correction. He was on the Plenary Session podcast describing it. I communicated with Andrew Gelman on this awhile back and it is still a little vague to me. I believe you don't have to do post hoc corrections like in frequentist analyses, because you should address it via your priors. So if you define the priors for your parameters/dist, then when you compare them, these priors should regularize your estimates. So, I would imagine if you used flat priors for all of these comparisons - you could be at risk for false discovery.

I would be curious to see what you have found in the literature on this topic and what package/function you using in R to do the Bayesian Spearman rank corr?

Thanks.
 
Last edited:
You must log in or register to reply here.