Can a naive hypothesis test determine that a coin is approximately fair?

#1
It's a very elementary example to determine fairness of a coin with repeated tosses and a hypothesis test, at least approximately. If the result of repeated tosses is fair, the coin is classified as fair.

However, a unfair coin, C, that can equally probably switch between two unfair states, p and (1-p) for heads respectively, may pass repeated tosses and be classified as fair.

Of course C may be examined for unfair switches, but there is no amount of tosses that may separate a fair coin and C.

Does this mean that naive hypothesis tests are dubious? On what condition may hypothesis tests be credibly employed?
 

hlsmith

Not a robit
#2
Well the law of large numbers states with infinite tosses the long run frequency should be 0.50 if its a fair coin. Given this, a coin in a not long run can by chance have 50 heads in a row. Though the probability is low. You can run a binomial test with a given probability for fairness and calculate the probability of getting that many heads or tails given the value you are examining.
 
#3
Well the law of large numbers states with infinite tosses the long run frequency should be 0.50 if its a fair coin. Given this, a coin in a not long run can by chance have 50 heads in a row. Though the probability is low. You can run a binomial test with a given probability for fairness and calculate the probability of getting that many heads or tails given the value you are examining.
Thanks. However, a binomial test would classify both a fair coin and the unfair coin C as fair due to the existence of switch.
 

Dason

Ambassador to the humans
#4
If you use a hypothesis that assumes a constant probability of success then sure if you have a magical coin that can switch probabilities then maybe it won't detect that. We can certainly devise a test that would be able to detect that but hypothesis tests can't assume things about the random variables that you don't tell it to assume.
 
#5
If you use a hypothesis that assumes a constant probability of success then sure if you have a magical coin that can switch probabilities then maybe it won't detect that. We can certainly devise a test that would be able to detect that but hypothesis tests can't assume things about the random variables that you don't tell it to assume.
Is there a rigorous qualification for sound uses of hypothesis tests?
 

Dason

Ambassador to the humans
#6
The tests you're probably familiar with all assume some sort of probabilistic model. If the assumptions of that model aren't met then it's possible that all bets are off. So... Learn what the model underlying your tests are.