Hi,

I am planning a follow-up experiment to a previous study and have a question about estimating sample size:

For the sake of simplicity, the previous study showed that Condition 1 had more accurate responses (64%) than Condition 2 (41%) and a paired samples t-test was significant (t(12)=3.43,p=0.005, d=0.9, two-tailed).

This finding has recently been challenged because the stimuli are somewhat confounded. My new experiment investigates whether a significant effect will be observed between condition 1 and condition 2 when new, more appropriate stimuli (after removing the confound) are used.

The problem is that when the new stimuli are used, the difference between conditions will *probably* be much smaller because we're supposedly removing a confound (e.g., ~10% rather than 23% difference). But we still hypothesise a sig. difference will be present. So how do I estimate the sample size needed to get an effect with the new stimuli? I think I have to estimate how much cohen's d would be reduced but it's almost impossible to do this precisely because I don't know how much the confound contributed to the initial effect.

Would it be sufficient to just estimate that cohen's d will be reduced (e.g., from 0.9 to 0.40) and then just do a power calculation based on that? (e.g., based on d=0.4, power=0.9, p=0.05, the estimate is 68 participants).

Many thanks for any insights!

Ryan

I am planning a follow-up experiment to a previous study and have a question about estimating sample size:

For the sake of simplicity, the previous study showed that Condition 1 had more accurate responses (64%) than Condition 2 (41%) and a paired samples t-test was significant (t(12)=3.43,p=0.005, d=0.9, two-tailed).

This finding has recently been challenged because the stimuli are somewhat confounded. My new experiment investigates whether a significant effect will be observed between condition 1 and condition 2 when new, more appropriate stimuli (after removing the confound) are used.

The problem is that when the new stimuli are used, the difference between conditions will *probably* be much smaller because we're supposedly removing a confound (e.g., ~10% rather than 23% difference). But we still hypothesise a sig. difference will be present. So how do I estimate the sample size needed to get an effect with the new stimuli? I think I have to estimate how much cohen's d would be reduced but it's almost impossible to do this precisely because I don't know how much the confound contributed to the initial effect.

Would it be sufficient to just estimate that cohen's d will be reduced (e.g., from 0.9 to 0.40) and then just do a power calculation based on that? (e.g., based on d=0.4, power=0.9, p=0.05, the estimate is 68 participants).

Many thanks for any insights!

Ryan

Last edited: