Chance performance in a binary response task

#1
Hello, I am confused by something and wonder if someone can help. With a task of 100 trials that are either TRUE or FALSE, where 80 are TRUE, what is the chance performance if a subject can respond "Yes" or "No"? Is this determined by the number of response options, or by the ratio of TRUE:FALSE?

I made a simulation that looks like it is 40% correct by chance and changes if the ratio changes (25% if 50:50), but was told flat out that chance is 50% by a friend who is better at math than me (because of the 2 response options). Who is right?
 

katxt

Well-Known Member
#2
If I am interpreting you correctly, the subject is just guessing. Of the 80 TRUE answers they will get 40 right by guessing yes. Of the 20 FALSE answers they will get 10 right by guessing no. Total 50 right by guessing. I think your friend is right.
 
#4
80/100 = 4/5 of answers are true, 1/5 of answers are false.
If the subject knows this, then their best guessing strategy is to guess true 4/5 of the time and false 1/5 of the time.
The best guessing strategy is guessing true all of the time, which inevitably gives 80% correct answers.
The expected rate of correct answers for your proposed strategy is 0.8*0.8+0.2*0.2 = 0.68 .

With kind regards

Karabiner
 
Last edited: