Estimating skill from small sample size question

[sheetspwns]

You have to assess how good someone is at (let's say for this example) shooting free throws. (binary outcomes)

You have player A who has made 60/100. That is all of the information and sample available.

You need to assess whether B is better than player A, and he starts shooting, and begins to make one after the other. How many consecutive Free Throws would he have to make, to convince you statistically that he is better than the person who made 60/100?

Obviously 1/1 is not enough, and obviously 60/60 for sure, but where is the breakeven? Is this able to be calculated using statistics? It is actually part of a sports betting analysis project I am working on, but converted the question to free throws here.

 

[joeb33050]

The probability of 6 consecutive free throws is just under 5%, a popular alpha. Given A % = .6.
Given A % = .6, binomial:
P 8 or more successes of 9 tries = 7%
P 9 or more successes of 10 tries = 4.6%
P 10 or more successes of 11 tries = 3%
Pick a %, pick "tries" and "successes".

 

[sheetspwns]

Let me simply further---If I shot X free throws in a row, my performance would be considered equally impressive as someone who shot 60/100. What is X?

 

[hlsmith]

Big question, before B starts shooting, did you hypothesize he was better, worse, or equivalent. If you wait until you start seeing data you are biasing the process.

 

[hlsmith]

So equal? If so, you have to create a threshold to define equal. You can't just say you are equal when a non-equivalency hypothesis fails. there are particular tests.

You need to look into, inferiority, equivalency, and non-inferior hypothesis testing and better define what you actually want to say or test.

Thanks.

 

[joeb33050]

x is 6, only 5% of the time 6 is pure chance-5% is an alpha populaire.