Ideal level of confidence/certainty

[AngleWyrm]

Staging the scenario
A casino offers you a gamble with a 1% chance of winning a try.

How many tries will it take to win at least once? The answer involves two variables, the chance of success each try and an allowance to be wrong in exchange for predictive accuracy. For this example, I choose 95% confidence, a willingness to be wrong once in twenty:

tries = log(chanceToBeWrong) / log(chanceFailureEachTry) = log(1/20) / log(99%) ≅ 300 tries

The question posed
But surely there must be a better way than to just arbitrarily select a level of confidence. It looks to me like an optimization problem, where tries are an expense to be minimized, and confidence spans across a range of 0..1

My first stab at it is to just rearrange the variables to solve for 1-confidence:

chanceToBeWrong = chanceFailureEachTry ^ numTries

But this is a surface of answers, like there's still some missing threshold, some cost/benefit crossover point(plane?)
Can anyone shed some light on how I might calculate the most appropriate level of confidence?

 

[Dason]

If you're looking to do a cost/benefit analysis then at the very least you would need to know how much you would win if you do in fact win and how much it cost to play. You haven't indicated knowing either of those so it seems a bit undefined at the moment.

 

[AngleWyrm]

Just assume cost is c and benefit is b
Do the same for any other information you deem necessary.

 

[fed2]

is this a 'geometric' random variable? sounds like it. that would give you the sampling distribution and other useful info.

 

[AngleWyrm]

Here's another version of the problem, for clarity.

I can tell you with absolute certainty what the outcome of a roll of two six-sided die will be — barring meta-events such as the cat batting one under the couch, or the scientist forgetting to record the results: It will be a number in the range 2-12. This is what 100% certainty offers, information on the full spread of outcomes without reference to their likelihood. It's a timeless and impartial image of the universe in all it's possible configurations.

But certainty can be spent on predictive insight, a focus of our gaze. The cost is an allowance to be wrong. The more confidence spent the more specific the view of the future, but also greater the chance it's just a fairy tale.



There seem to be two conflicting forces at play:

• Correctness, where higher proportion of possible outcomes is better, reducing the chance of being wrong
• Accuracy, where a lower proportion of possible outcomes is better, painting a more specific prediction.

And so it seems to be a measure of the importance of Accuracy vs Correctness that would determine choice of confidence level. This seems to be a form of cost/benefit analysis between accuracy and correctness.

Question
How would you approach deciding which level of confidence to select?

 

[fed2]

add two columns to ur table. one called 'cost', this is the net cost of being wrong given the predication. one called 'benefit', this is the net earnings for being right on a given prdiction. Note that net earnings/losses may include some costs associated with the number of outcomes allowed for in the prediction, or anything else you like.e

calculate the expected earnings, for each predication, as 'proportion'*benefit - 'cost'*(1 - proportion).

choose the prediction with the greatest expected earnings.

repeat until millionaire, the focus of our gaze.

 

[noetsi]

I think another thing to consider is probabilities might change and your cost benefit calculations might as well. Your willingness to take a risk might be different at 20 than 75...

 

[fed2]

I think another thing to consider is probabilities might change

No im afraid that is incorrect. According to anglewyrm, they are "timeless and impartial image of the universe in all it's possible configurations.".

 

[AngleWyrm]

In case there remains some doubt, here are all the possible outcomes of rolling a pair of six sided dice:


There are 6 × 6 = 36 possible outcomes.

 

[fed2]

i see this table is timeless indeed! But how do you know that is impartial?

 

[AngleWyrm]

i see this table is timeless indeed! But how do you know that is impartial?

Dice outcomes appear in equal proportions. That is part of the definition of dice, coins, and random number generators.

If you need something other than normal behavior, you'll either have to declare the equality of outcomes property of the dice/coin has been altered and express that different proportion, or measure a sample population for the outcomes it produced.

Here is an example of unequal proportions: Flip five coins, did you get a head? My answer is yes, and I'm willing to wager being wrong about that once out of every 32 times the five coins are flipped. 1 - 1/32 = 97% confidence in the outcome, or to put it another way, I plan on being wrong about 3% of the time in exchange for the ability to predict the future.

 

[fed2]

my cat is still confused about impartial-ness. can you explain again in a cat friendly way?

 

[AngleWyrm]

When rolling two six-sided dice there are 36 possible equally likely outcomes.

Let's say for example we want to know more about getting a result where the dice total 10.
There are 3 outcomes in the set of 36 possible outcomes that have 10 pips showing: [ (6,4), (5,5), (4,6) ]

So it amounts to playing spin the bottle with the needle pinned to the center of this pie chart


Here's what the full set of totals looks like in a spin-the-wheel arrangement

 

[fed2]

what about if you use a 20 sided dice with a +1 savings throw?

 

[AngleWyrm]

With 1 6-sided die there are 6 outcomes
WIth 2 there are 6x6 = 36 outcomes
With 3 there are 6x6x6 = 216 outcomes
To generalize, sidesPerDie^numberOfDice = number of outcomes
Here's how to find the count of outcomes for each possible total

 

[Dason]

We all understand how to get the total number of itchiness and probabilities for dice rolls. You don't need to keep posting that. You're definition of accuracy and correctness seem to be just the opposites of one another so it's not like you're asking to optimize two quantities. To me the issue is just that you haven't defined the issue well enough. If there is a cost and benefit then you can analyze specific situations to try to maximize payout/risk/whatever. You haven't really defined those well enough and that's why I don't think you're getting good answers.

Maybe try to find up with a specific example with specific costs/benefits.

 

[AngleWyrm]

You're definition of accuracy and correctness seem to be just the opposites of one another so it's not like you're asking to optimize two quantities...If there is a cost and benefit then you can analyze specific situations to try to maximize payout/risk/whatever.

Map accuracy to cost and correctness to benefit (or visa versa as you please), then see if you can add something constructive to the conversation.

 

[Dason]

I'll give you one more chance for you to add something to the conversation. Like I said - give a specific example because regardless of what you think - you're being too vague. Trust me when I say that a lot of people here are more than knowledgeable when it comes to this stuff and can help you if you put in some effort. You tend to be way too vague and even offended at the idea of adding specifics to help illustrate what you want to do. I'm trying to help by suggesting that you give a specific example. If you don't want to then that's fine but don't be surprised if nobody helps you.

 

[noetsi]

No im afraid that is incorrect. According to anglewyrm, they are "timeless and impartial image of the universe in all it's possible configurations.".

Then he understands reality and human nature differently than I do. God is changeless nothing else is.

 

[AngleWyrm]

...give a specific example

Staging the scenario
A casino offers you a gamble with a 1% chance of winning a try. How many tries will it take to win at least once? The answer involves two variables, the chance of success each try and an allowance to be wrong in exchange for predictive accuracy. For this example, I choose 95% confidence, a willingness to be wrong once in twenty:
The question posed
But surely there must be a better way than to just arbitrarily select a level of confidence. Can anyone shed some light on how I might calculate the most appropriate level of confidence?