I missed bruin's comment so this is a long winded development of their point.

One thing that has been stressed more in the literature is the fundamental difference between statistical difference and substantive difference. The former involves the classical p values and null hypothesis. What this is really testing is whether the results, the effect size is likely due to random error in measurement. For many years I believe there was tendency to assume if something was statistically signficant the effect size was substantively important (I hear this on my job all the time).

But that view has been challenged a lot recently. Just because an effect size is statistically signficant does not mean it matters in any meaningful way. If your test is powerful enough you can have very small effect sizes, say differences in means before and after, and they can stil result in very small p scores signifying statistical signficance.. Essentially everything is signficant statistically when you have enough power.

The key then is to make a decision that given this effect size, the difference in means here, is real does it really matter? And that is a a decision for SME not statisticians based on context, the nature of the phenomenon etc.