P value of a statistical test

Status
Not open for further replies.

pmcn

New Member
#1
If the p-value of a statistical test is .038 and the level of significant chosen for the hypothesis tst if .05 then.

I believe the correct answer is "do not reject" since .038 is less than .05 -

can anyone tell me if this answer is correct?
 
#2
If your observed value is less than your significance criterion, you reject the null hypothesis. If your observed value is greater than your significance criterion, you fail to reject the null hyothesis.

0.038 < 0.05; so you reject the null hypothesis.
 

TheEcologist

Global Moderator
#3
If the p-value of a statistical test is .038 and the level of significant chosen for the hypothesis tst if .05 then.

I believe the correct answer is "do not reject" since .038 is less than .05 -

can anyone tell me if this answer is correct?
In addition to the remarks of jamesmartinn.

When doing any statistical test, you can view the p-value as the "chance" that the null hypothesis is true. Now as a convention in most sciences we say that if the "chance" or p-value drops below 0.05 (or 5%) we can safely assume that it is not true.

In other situations and in esp. medical science the "critical" P-value is lower (say 0.01 percent, because peoples lives could be at stake). So we are (more) sure we dont reject the null hypothesis incorrectly.

So basically just remember that the p-value represents the "likelihood" or "chance" that the null hypothesis true.
 

JohnM

TS Contributor
#4
"you can view the p-value as the "chance" that the null hypothesis is true"

That's actually a common misconception...

The p-value represents the probability of obtaining the value or larger value of the statistic, if in fact the null hypothesis is true.

In other words, it is representative of the weight of evidence in support of rejecting the null --> if the p-value is small, the evidence against the null is strong, if the p-value is large, the evidence against the null is weak.

The null hypothesis is either true or not (it is merely a theory phrased in a way that allows it to be tested), and does not have a probability associated with it....Therefore the p-value is definitely not the chance that the null is true.
 

TheEcologist

Global Moderator
#5
"you can view the p-value as the "chance" that the null hypothesis is true"

That's actually a common misconception...

The p-value represents the probability of obtaining the value or larger value of the statistic, if in fact the null hypothesis is true.
:yup:

That’s why I used the quotation marks when writing “chance” above.

It's even worse: the p-value represents the probability of obtaining a particular value or a greater value of the statistic based on a parametric model you parameterized with your sample values. Therefore it represents the fraction of a theoretical population that is equal to or greater than the value of interest.

However I have found that when explaining all that to people who really hate stats, it becomes even more confusing. In the beginning all the terms and definitions seem overwhelming so I simply use this little "trick" and it really seems to help people remember it (ergo pass the test). Although it's not completely true, I always think that once they get more adept in stats they will figure it out. I view it as the lesser or two evils.

However you are probably far more experienced in teaching stats so if you have a good argument why I should not use this trick for people just beginning to learn stats I’m all ear.

thanks,

Edit:

However after talking it over with a peer I found that I really should refrain from using the word chance.
 
Last edited:

JohnM

TS Contributor
#6
Usually I explain it like a court case - the null hypothesis is the defendant's claim of innocence, and the p-value is the weight of evidence against the claim. If the p-value is small, then it is unlikely that the defendant is innocent.
 

Martingale

TS Contributor
#7
:yup:

That’s why I used the quotation marks when writing “chance” above.

It's even worse: the p-value represents the probability of obtaining a particular value or a greater value of the statistic based on a parametric model you parameterized with your sample values. Therefore it represents the fraction of a theoretical population that is equal to or greater than the value of interest.
...
Obviously this is over complicating things.

Why can't you just say it like this (though I wouldn't use the word larger)...

The p-value represents the probability of obtaining the value or larger value of the statistic, if in fact the null hypothesis is true.
 

TheEcologist

Global Moderator
#8
Obviously this is over complicating things.

Why can't you just say it like this (though I wouldn't use the word larger)...
I think, JohnM way of referring to the p-value as support or evidence for the null hypothesis would work best.

I am of course not talking about the official definition of the P-value but just a trick to help a person remember which hypothesis should be rejected.
 

JohnM

TS Contributor
#9
Many times we have to "over-simplify" something in order for non-statisticians to "get it."

The problem arises when they try to pass the over-simplified explanation onto someone else, and the misconception spreads....
 
#10
For many practical settings I think this would say it:

The p-value represents the probability of falsely "proving" that the treatment works when it actually doesn't.

Like the other simplified explanations this is NOT a definition, but it helped me when I was first trying to understand the meaning of the prober definitions.

Saying the treatment works when it actually doesn't also means making a type 1 error.
 

JohnM

TS Contributor
#11
I don't know. The more I think about it, the more I think we need to get back to the fundamentals of what we're really doing, explain it in that fashion, and if you don't get it, sorry, but that's the way it is.

Sometimes we treat people as if they have the "right" to understand probability and statistics, but that's like me claiming I have the "right" to understand theoretical physics, and clearly I don't..

...and I don't expect anyone to be able to explain it to me in terms I can understand....

we could go on forever, so I think I'll close this post...
 
Status
Not open for further replies.