Testing if the averages between two independent data are closer compared to two other data

#1
Suppose that I sample 4 independent datasets from different, 1D normal distributions: data1, data2, data3 and data4.

I want to test if data1 and data2 have closer means compared to data3 and data4, e.g if

|m_1 - m_2| < |m_3 - m_4|


I define the following variable Zij:



if m_i - m_j > 0 (do a ttest): Z_ij = x_i - x_j

elseif m_i - m_j < 0: Z_ij = x_j - x_i

where x_i ~ N(m_i, s_i), x_j ~ N(m_j, s_j)



Then, Z_ij ~ N(|m_i - m_j|, sqrt(sigma_i^2+sigma_j^2))`.

At the end I can compare Z_12 and Z_34 with another ttest.

Do you think that to be correct if then I correct the final alpha for multiple testing ?

Thank you in advance
 
Last edited:
#2
If you are comparing the same response and datasets 1 through 4 differ in the way of an explanatory variable. You could do one way anova maybe. and do a tukey adjustment for multiple comparisons
 
#3
If you are comparing the same response and datasets 1 through 4 differ in the way of an explanatory variable. You could do one way anova maybe. and do a tukey adjustment for multiple comparisons
Hello, thank you for answering. Yes It is my main approach for now.
The pb is that we are testing the quality of the processing based on non-significant differences after processing while for non-significant results we cannot conclude anything (we don't know the false negative rate to estimate our accuracy)...
Also, little average differences tend to be always significant if data goes to infinity which doesn't highlight the fact that the processing may importantly reduce the distance between the group averages even if a small distance remains ...
 
Last edited:
#4
I finally found a solution, I compute the two difference datasets such as explained before and then the confidence interval on the ratio of means and check if it is comprised between -1 and 1. :)