Time Series for Binary Events of Unknown Dependence

#1
Short version: You have a series of binary events, presumed independent but not known for sure. A subset of these series show all successes or all failures. If the subset was the entire dataset, it would be statistically improbable (which would force us to revisit the assumption of the underlying probability or the independence of events). But the subset is not the entire dataset. How do you calculate the probability of that subset occurring, for a given dataset size?

Long version (with my meandering attempts to figure this out myself, unsuccessfully):

I don't have a particular problem I am looking to solve, I am more interested in understanding how to handle a certain class of problems.

Let's say I flip a fair coin 7 times in a row and it comes up heads each time. Each flip is independent. It can be easily calculated that the probability of 7 heads in a row is about 0.8%.

It is also pretty straight forward, with binomial probability, to be able to look at a data set and see if a sample is consistent with previous assumptions. Let's say a free throw shooter makes 65 shots in 100 tries, and the prior probability for that free throw shooter was 50%. In this scenario, it's much more probable that the free throw shooter improved their shooting, than they just got lucky.*

Which brings me to the class of problem I'm trying to solve. The "hot hand" problem. The hot hand is when an athlete goes on a hot streak. In basketball, it may be a player making 8 consecutive field goals. The idea is as follows: the player makes a few in a row, and gets a boost of confidence, which makes the subsequent shots more likely to go in. A Lindy effect, if you will. Of course, this is called a fallacy for good reason--with hundreds of athletes across dozens of sports, someone is bound to be "hot" at any given time.

But... but! Can we truly say that basketball field goal attempts are truly independent? It's easy to speculate that some time sequences will be correlated. Did the athlete's newborn wake up every hour the previous night? Has a nagging injury finally healed? Did the coach rest the player the previous game, so they are fresh?

To put it in mathematical terms, let's say I have an event with an assumed 50% probability, let's say a free throw shooter in a controlled environment. Let's say we are unsure if the events are independent. Finally, let's say the shooter makes 7 shots in a row.

If the # of attempts is exactly 7, we would be forced to concluded that either the shooter has more than 50% odds of making a basket, or the events are dependent. But if the # of attempts is 100,000,000, making any 7 in a row is pretty meaningless.

So my question is how to approach that mathematically. One way I considered doing that, is that the chance of 7 in a row is assumed 0.8%. If we have 100 "7 shot attempts", the odds of getting at least 1 set of 7 in a row is about 45%. But how do I define what a "7 shot attempt" is? Here were a couple things I tried



7 shots.PNG

The problem with the first definition, is the "7 shot attempts" are clearly not independent. For example, missing attempt #2 invalidates both "7 shot attempts". If you used this method, you would need 106 shots to get 100 "7 shot attempts." I simulated this with a monte carlo engine. With 106 shots of 50% probability, completely independent, the chances of 7 or more in a row was about 20%, well short of the 45% presumed above.

The second definition is equally problematic. It would ignore the fact that making 4 through 10 counts as 7 in a row. Again, it fails the monte carlo simulation. If I use 700 shots to get 100 "7 shot attempts", the odds of 7 or more in a row was about 75%, well above the 45%.

So both of those approaches are clearly incorrect, demonstrated both logically and through empirical simulations.


So here is my question - how do you handle this class of problem? How do you know if a sequence of successes or failures in a binary outcome is an outlier (assuming independence) or if it is statistically likely given the sample size.


*depending on how many 100 shot samples we have. If he does this every hour for years, eventually >= 65 will happen by luck alone.
 
Last edited:
#3
Here's another interesting, related topic. Let's say our athlete makes 45/100 shots. They then hire a coach and make 55/100 shots. Each shot is independent, but the second 100 shots have a 55% chance, versus the first set that has a 45% chance. The chance of making 55 or more shots in a sample of 100 while shooting with a 45% odds is only 2.8%. Let's state that the coach did in fact improve the performance of the athlete.

Now let's say an analyst looks at this data. They observe the athlete has made 100/200 shots for a 50% chance. They also notice that the athletes performance is different from the first half of the sample versus the second.

Capture.PNG

To the analyst, it is possible the athlete is a 50% shooter, and just was unlucky in set 1 and lucky in set 2.

There doesn't appear to be a way for this analyst to determine whether the athlete improved with a 97.2% probability, or was always a 50% shooter and simply had an unlucky streak followed by a lucky streak.
 
Last edited:

fed2

Active Member
#4
honestly just use 'hominid assisted empirical bayesian' method: plot it and if it seems like the odds changed to you then i guess so. you know more about the players/game than any computer anywayh.
 
#5
honestly just use 'hominid assisted empirical bayesian' method: plot it and if it seems like the odds changed to you then i guess so. you know more about the players/game than any computer anywayh.
Ha! Took me a second to understand. I was hoping to find a more definitive answer, but the more I get into these things the more I see judgment calls and the less I see definitive answers.

Some times I wonder about the way probability is taught. Combinations, permutations, binomial distributions... all the coursework (that I went through, a B.S in Engineering) had concrete answers. In the real world, I’m finding concrete answers few and far between.
 

fed2

Active Member
#6
You may also want to try a 'runs test' though. thats the standard test for independance out their.

Yes, I work in biology and I can tell you that, in my experience, scientists can outsmart stats. The amount of math that it takes to even come close to experiened judgement is pretty astounding. Sure it is possible to build chess computer and the like better than people, but youd still be a fool to ignore the recommendations of kasparov,