Understanding correlations in order to test speed-accuracy trade-off in experiments

#1
I understand that in experiments where the measures of interest are speed (reaction time) and accuracy, for an effect to be valid one needs to make sure there is not a speed-accuracy trade-off (SAT). Namely, as reaction time gets quicker for a condition, accuracy should also be better for that condition.
Here are my questions:

1) If there is no significant correlation, but the trend looks right is this still not a speed-accuracy trade off?

2) In my current experiment, responses use two hands (one at a time). I ran an ANOVA and found no interaction for Hand, so I collapsed that particular condition. When I check speed and accuracy for each condition, there is no significant correlation. But when I separate the hands again, each hand shows a significant correlation i.e. good evidence that there is not a SAT. But how can the hands together show no SA correlation but separately they do?!

3) Participants scored over 80% accuracy in all my conditions - does this mean there is no SAT or need to test correlations?

Thank you very much in advance!
 
Last edited:

Karabiner

TS Contributor
#2
Re: Understanding correlations in order to test speed-accuracy trade-off in experimen

Please describe the topic of your study and the research question,
and how the performance task looked like, and what was actually
measured, and how large your sample size was. And how your p values
actually looked like - "non-significant" can mean anything between
0.050 and 1.0 . "Signficant" can mean anything from 0.049 to 0.00000...1.

I understand that in experiments where the measures of interest are speed (reaction time) and accuracy, for an effect to be valid one needs to make sure there is not a speed-accuracy trade-off (SAT).
Performance tests really have to demonstrate a tradeoff of exactely 0.000
in order to be valid? In how many performance tasks involving speed and
accuracy can this be expected? This not a rhetorical question.

1) If there is no significant correlation, but the trend looks right is this still not a speed-accuracy trade off?
Non-Significance doen't mean much if the sample size is small. In that case,
power to detect signficant associations is low, even if associations exist
(in the population). How large was your correlation coefficient?

2) In my current experiment, responses use two hands (one at a time). I ran an ANOVA and found no interaction for Hand, so I collapsed that particular condition.
Wheter the non-significance of the interaction is meaningful, depends
on the sample size. And which variables were involved as UV in addition
to "hand", and what was/were the DV?

When I check speed and accuracy for each condition, there is no significant correlation. But when I separate the hands again, each hand shows a significant correlation i.e. good evidence that there is not a SAT. But how can the hands together show no SA correlation but separately they do?!
I don't know much about what you did, and descriptive statistics
and p-values are completely missing, so this cannot easily be
answered.

3) Participants scored over 80% accuracy in all my conditions - does this mean there is no SAT or need to test correlations?
What was the mean, median, range and standard
deviation for your accuracy measure? What were the
according descriptive statistics for your speed measure?

With kind regards

Karabiner
 
#3
Re: Understanding correlations in order to test speed-accuracy trade-off in experimen

Hi Karabiner! Thanks very much for your responses!

The topic is looking at the effect of variable A versus B (perceptual stimuli) on reaction time (measured in milliseconds) and accuracy (proportion of trials correct)
Two hands were used. Pressing button 1 for variable A1 and variable B1 and button 2 for variables A2 and B2.
Within-participants design
Two sessions: in one session, participants pressed button 1 for A1 and B1 and button 2 for A2 and B2, then vice versa in session 2 (all counterbalanced)
Button 1 was pressed by left hand, button 2 by right hand
Sample size was 31

The pearson's r correlation between reaction time and accuracy for the condition of interest, e.g. A1 for right hand was -0.41 p = 0.023 and for left hand -0.36, p = 0.044 (not exact figures-I don't have them here :( but I think they were this) but when I put left and right hand together and just looked at A1 (hand condition collapsed) the correlation between reaction time and accuracy is -0.30 p = .147

I can't explain this? Why would the significance disappear? (ie. >.05)
 
Last edited:
#4
Re: Understanding correlations in order to test speed-accuracy trade-off in experimen

Performance tests really have to demonstrate a tradeoff of exactely 0.000
in order to be valid? In how many performance tasks involving speed and
accuracy can this be expected? This not a rhetorical question.

I'm afraid I don't understand this response :( the SAT is just something we're supposed to check for in psychophysics...
what is a trade off of 0?

What was the mean, median, range and standard
deviation for your accuracy measure? What were the
according descriptive statistics for your speed measure?

there are descriptives for each condition: A1 left and A1 right were 95% and 97%, B1 left and right around 83%