Null Hypothesis Rules: Can no difference be the alternative hypothesis?

#1
For his dissertation a colleague has developed a software tool and has the expectation (hypothesis) that people can complete the task with the software tool without taking longer than the older, manual method. He has to write up his hypotheses with null and alternative hypotheses. He asked me if he could use no difference as the alternative hypothesis and to be honest despite being fairly familiar with statistics i could not think of a case where i had seen it.
By the definitions, the null hypothesis is always, or includes, no difference. It is also the general rule that you never accept the null, you can just fail to reject it. That makes hypothesizing to fail to reject the null a bit odd as well.

Has anyone come across a similar situation and had to phrase the question in terms of null and alternative hypotheses? In practical research one would just write that they expect no difference, run the test (with significant power to find a difference) and if no significant difference was found then say that.
 

Karabiner

TS Contributor
#2
By the definitions, the null hypothesis is always, or includes, no difference.
Not necessarily. The null hypothesis ist the hypothesis TO BE NULLIFED
(cf. Fisher). It must be a point hypothesis (at least in the "two-sided" case).
So you could test mean1 - mean2 = 2.6557 just as easy as mean1 - mean2 = 0.00000000.
It is also the general rule that you never accept the null, you can just fail to reject it. That makes hypothesizing to fail to reject the null a bit odd as well.
Yes. And with very small samples it would also be very easy to maintain a
(wrong) null hypothesis, because small samples do not provide much empirical
evidence against the null. But a non-significant result just because of lack of statistical
power would of course not suppport the claim that theres's no difference in
the population.

Do a little research for "equivalence testing" or "noninferiority testing", e.g.
http://www.graphpad.com/guides/prism/6/statistics/index.htm?testing_for_equivalence2.htm
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3019319/

With kind regards

K.
 
Last edited:

rogojel

TS Contributor
#4
has the expectation (hypothesis) that people can complete the task with the software tool without taking longer than the older, manual method.

He has to write up his hypotheses with null and alternative hypotheses. He asked me if he could use no difference as the alternative hypothesis.
Hi,
in this particular case it seems to me that the traditional approach should not be abandoned. I expect that the goal of the sw is to make the task easier. Taking this for granted (which would be equivalent to making this the null hypothesis) seems to me to be unfounded - it is like assuming that the project was successfull without first proving it. A further problem is that we never prove the null hypothesis, so this would be equivalent to making the statement that the method is better by default and expecting it to be disproved -I would personally find such an approach very problematic.


Your colleague could assume, as Karabiner said, that the difference is greater then a given limit, with the null hypothesis being, that the difference is less then the limit, but this is still within the traditional framework.

regards
rogojel
 

noetsi

Fortran must die
#5
If t1 is the time to complete the work with existing software and t2 the time with the new software:

Why wouldn't the alternative hypothesis be t1-t2 >0
And the null be t1-t2 not greater than 0?

Probably missing something really basic here:p
 

Karabiner

TS Contributor
#7
And the null be t1-t2 not greater than 0?
But that would not solve the problem that with small sample
size that null can hardly be rejected, only if the effect is large.
Pragmatically, it is the same test as the two-sided test, only
that the p-value is halved.

With kind regards

K.
 

rogojel

TS Contributor
#8
But that would not solve the problem that with small sample
size that null can hardly be rejected, only if the effect is large.

With kind regards

K.
hi Karabiner,
I am wondering why that would be a problem? It is like postulating the success of the experiment and expecting failure to be proven. If the null were that there is an effect and I did not collect enough data, I would still be able to say, that the present experiment did not disprove my assumption that I was successfull. I think this is precisely the way all quacks argue, and should not be accepted as a rigorous argument.

Do I miss something?

regards
rogojel
 

Karabiner

TS Contributor
#9
I am wondering why that would be a problem?
I was referring to the OP's actual problem, who seemingly wants
to be able to claim non-inferiority based on empirical evidence.
Non-rection of t2-t1 = 0 only due to insufficient power would not
provide such evidence.
If the null were that there is an effect
There are no such null hypotheses. Null hypotheses are exact
statements, like t1-t2=0 or t1-t2 <=2.87937397 . Unlike what
is often taught, working hypotheses ("H1") and statistical null
hypotheses are not on the same conceptual level, and not simply
interchangeable.

With kind regards

K.
 

noetsi

Fortran must die
#10
But that would not solve the problem that with small sample
size that null can hardly be rejected, only if the effect is large.
Pragmatically, it is the same test as the two-sided test, only
that the p-value is halved.

With kind regards

K.
I thought the issue was creating the correct null. If the issue is power and you can't increase sample size or reduce the SE you have problems...