Bayesian Posterior: SD = SE?

hlsmith

Less is more. Stay pure. Stay poor.
I was curious if the standard deviation in a Bayesian posterior is equal to the standard error?

Dason

A standard error is literally just the standard deviation of the sampling distribution for whatever estimator you're using.

So I'd say it's analogous but isn't exactly the same.

Jake

Technically the answer to this is always no, since these two quantities have totally different conceptual meanings -- one is the SD of the marginal posterior distribution for a parameter, one is the SD of the sampling distribution of the estimate of the parameter.

With that said, we might still ask if there are cases where these two quantities numerically coincide, i.e., $$SD_{posterior} = SD_{sampling}$$. To focus the discussion, consider the case of the regression coefficients in a regression model (i.e., the $$\beta$$s). So the answer here is, I believe, "in some special cases the two coincide, but not in general or even in the most common real-world cases."

For one thing, if the Bayesian model in question involves non-flat priors, then the two will not coincide, except by accident. Now, if one uses a flat prior on $$\beta$$ AND we consider the residual variance $$\sigma^2$$ to be fixed/known for both the Bayesian and classical models, then I think that the two do coincide, since the posterior distribution $$\beta | y$$ is then Normal and I think numerically identical to the Normal sampling distribution of $$\hat{\beta}$$. But of course we usually consider $$\sigma^2$$ to be unknown and something we need to estimate. In that case, the posterior $$\beta | y$$ is no longer Normal, since it's marginalized over whatever the distribution for $$\sigma^2$$ is. But the sampling distribution of $$\hat{\beta}$$ is still Normal. So the SDs of these two distributions shouldn't match except by accident. However I think it's true that $$\beta | y, \sigma^2=\hat{\sigma^2}$$ -- the posterior that conditions on $$\sigma^2$$ being the point estimate $$\hat{\sigma^2}$$ from the classical model -- is still Normal and matching the sampling distribution as before. Whatever that buys you. Anyway, some actual statistician type person should check over all this.

hlsmith

Less is more. Stay pure. Stay poor.
Thanks. I had some of these thoughts as well, that is why I posted the question. I would imagine there would also be a given model specification component come into play as well.

In the referenced paper they were using uniform priors and I believe empirical for model probability.