So this popular minecraft speedrunner has been accused of cheating with pretty solid proof. Later, the speedrunner(Dream) had released the rebuttal, which was extremely flawed(in my opinion). Can someone help me look over the math(in here: https://drive.google.com/file/d/1yfLU...) and tell me what they think of it?
Dream's Speedruns
- Thread starter 77Panda
- Start date
just a general comment on these issues of 'cheater' catching. Need to keep in mind that the important part is the 'positive predictive value' ie the chance that they are cheating given the observed outcome, rather than the observed outcome given the evidence, which is what is often tabulated. I think that sometimes gets missed in these things. This report is pretty lengthy, maybe you could break it down or give a small case in point.
https://towardsdatascience.com/the-prosecutors-fallacy-cb0da4e9c039
https://towardsdatascience.com/the-prosecutors-fallacy-cb0da4e9c039
I'm finding that the previous document(the accusatory one) is far more accurate than the rebuttal, (btw the accusatory one is here- https://mcspeedrun.com/dream.pdf).
Here's my thought process so far:
On the trades, the stopping rule on individual runs does not affect the distribution since the data was pooled across multiple streams, including unsuccessful runs. Only the stopping rule on streams itself affects the likelihood function, which causes only the last record stream to be not exchangeable, and that’s accounted for in the first analysis. There's another error that I found while reading the rebuttal: the way they implemented it, giving 8 pearls in a barter has probability 0 while the comment states it should be 4-8 pearls.
1. This is not a situation where you can go “there are different but both plausible assumptions which lead to different answers”—there are clear mathematical and computational errors in one paper.
2. “Expert” in one field =\= expert in another. An astrophysics PhD is not an expert in statistics. People seem to have this idea that all fields with mathematical training are essentially interchangeable.
3. Do you really think that as long as you have someone with a PhD in any mathematical field disagreeing with a statistical analysis, then statistical analysis is pointless? Because there is ALWAYS at least one PhD who disagrees, ESPECIALLY if you’re allowed to pay them to
The 19 page document written by the PHD assistant professor actually contains a multitude of mathematical and coding errors that can be spotted. Such as using the 4-7 pearl drops instead of 4-8, including the 5 previous streams etc. When talking about reddit, a known verified user known as u/mfb- has taken apart this paper. Before calling them anonymous, they are verified because they had to submit their credentials to the moderator of the subreddit to get the role, so if you want credibility, he has more than what the assistant professor has. (Also the person that confirmed it was a discord chat mod, not an actual part of the moderator team)
Here's my thought process so far:
On the trades, the stopping rule on individual runs does not affect the distribution since the data was pooled across multiple streams, including unsuccessful runs. Only the stopping rule on streams itself affects the likelihood function, which causes only the last record stream to be not exchangeable, and that’s accounted for in the first analysis. There's another error that I found while reading the rebuttal: the way they implemented it, giving 8 pearls in a barter has probability 0 while the comment states it should be 4-8 pearls.
1. This is not a situation where you can go “there are different but both plausible assumptions which lead to different answers”—there are clear mathematical and computational errors in one paper.
2. “Expert” in one field =\= expert in another. An astrophysics PhD is not an expert in statistics. People seem to have this idea that all fields with mathematical training are essentially interchangeable.
3. Do you really think that as long as you have someone with a PhD in any mathematical field disagreeing with a statistical analysis, then statistical analysis is pointless? Because there is ALWAYS at least one PhD who disagrees, ESPECIALLY if you’re allowed to pay them to
The 19 page document written by the PHD assistant professor actually contains a multitude of mathematical and coding errors that can be spotted. Such as using the 4-7 pearl drops instead of 4-8, including the 5 previous streams etc. When talking about reddit, a known verified user known as u/mfb- has taken apart this paper. Before calling them anonymous, they are verified because they had to submit their credentials to the moderator of the subreddit to get the role, so if you want credibility, he has more than what the assistant professor has. (Also the person that confirmed it was a discord chat mod, not an actual part of the moderator team)
The important thing is that the interested parties agree a-priori (before hand) how the stats are going to be counted, this is extreeeeeeeemly important when in any sort of adversarial context. Its arguably more important than 'doing it right'. What outcomes will be considered 'cheating', which will not? You can pay people to do whatever, but they have to do it at certain times is the point. After the fact, its gotta certain types of debate should be closed for discussion.
Yeah you got 'errors' then you got 'errors' right. If its a flat out 1=2 type computational error, that's fair game for criticism in my view. If they aren't willing to consider those points, you should just save your breath.
Im not really a minecrafter so i cant really get into bartering rules and stuff.
Yeah you got 'errors' then you got 'errors' right. If its a flat out 1=2 type computational error, that's fair game for criticism in my view. If they aren't willing to consider those points, you should just save your breath.
Im not really a minecrafter so i cant really get into bartering rules and stuff.