If you discover an effect size you believe are real, what does it matter what the CI is? Alternately if the CI (or p value etc) suggests that the result is real but it is tiny what is the impact practically of that? Statistics and substantive value are entirely different. The only thing statistics can tell you is if the results are likely to be random error not if they matter. That is outside the realm of statistics a point increasingly being emphasized in statistical texts.

I agree entirely. My point is that objective thresholds are not determined by statistics but by impact and professional judgement.

Part of my comment here is also to Joey

Even if we determine the margin of clinical equivalence, we don't know about the variation in our sample. So a 103-unit difference is as vague as an average without standard deviation. Can somebody tell if a difference is significantly greater than zero or not, if he doesn't know the standard deviation?

Now, in this CI model, they have replaced that "zero" line with a "band" with a non-zero width. We should check if the CI has crossed that band or not. If it dooesn't cross the band (both bound of CI greater or lesser than the band, simultaneously), we can conclude that our average 103 unit difference is for sure greater than the band (so there is practical difference).

If the CI crosses the band, there are two form:

either it totally crosses the band, as Dason suggested (one CI bound below the band, the other beyond it). Here we can be sure that it is not at all practically significant. Or half of it crosses the band, and the other part remains within the band (the problem of Joey). Now, we can't conclude

**for sure** that if it is practically significant or not. Although we can still become quite subjective and discuss the topic, but now this method of CI can't help us.

-------------

I agree that those objective thresholds are not quite statistically determined (although I have seen a couple of papers, in which the threshold was determined using stats). But once the threshold is found based on consensus or Delphi method (or even statistics if possible), we can use that for a semi-objective analysis.