paired sample t-test for repeated measures in 1 subject

#1
I am conducting a psychology experiment. In my pilot study I analyze responses of one subject which ran through two conditions.

The task is to predict outcomes based on previous outcomes. On every trial the subject tries to predict the outcome as good as possible and learns to adapt the prediction through the prediction error (difference between predicted and actual outcome). In one condition (say the easy one), outcomes change slowly, in the other condition (the hard one) outcomes change very often and the subject has to frequently update the prediction.

Data could look like this:

Prediction Error
cond1/ cond2

10 / 44
24 / 31
54 / 3
34 / 20
2 / 20
4 / 15

I want to compare the mean of the prediction error of the two conditions.
As the subject runs through both conditions, one would use a paired sample t-test. However, the trials of both condition are not strictly related (in the sense that one is a pre- and the other is a post-treatment test). The outcomes are pseudo-randomly generated and therefore, outcomes in the conditions are not strictly paired.

My question is: do I have to use a paired sample t-test and if so, why not an independent measures t-test.

Any suggestions are welcomed!

Monstera
 
#3
Are the measurements in each column correlated? Is each column the same study (spread in time) for the same subject?
The measurements in each column should be uncorrelated as the outcomes are independently generated in each condition (and for each subject).

Regarding the second question: every column is one trial which is evenly spread in time.
 

staassis

Active Member
#4
Ok... Each pair of measurements in correlated because we are talking about the same subject... If the data are normal in each column, you should run the paired-sample t-test. Otherwise you should run the paired-sample Wilcoxon test.
 
#5
Thank you for your reply.
I understand that each pair is correlated as the same subject is performing the task.

I am now trying to understand the core idea behind the paired-sample t-test. My main difficulty is that the outcomes of the two conditions are randomly generated. I get that subject attributes have an influence on performance in both blocks (and therefore that they are correlated) but I don't get why every trial pair should be dependent as such. Would a different order of the stimuli still lead to the same result in a paired sample t-test?
 

staassis

Active Member
#6
Did not understand your question. Please rephrase.... Sure, the measurements are random in both columns. Otherwise there would be no testing. But they are co-dependent in each pair.
 

Karabiner

TS Contributor
#7
As the subject runs through both conditions, one would use a paired sample t-test.
No, you just have one subject who produces several responses.
Seemingly, there's no reason to perform a paired test, since
responses are not paired. I don't know how much sense it
makes to perform a significance test here (what do you want
to generealize to?), but independent samples tests look
appropriate.

With kind regards

K.
 

Karabiner

TS Contributor
#8
If the data are normal in each column, you should run the paired-sample t-test. Otherwise you should run the paired-sample Wilcoxon test.
Not exactely. An assumption of the paired t-test is that the
differences should be from a normal population (at least, if
n is small), not the column variables. It is exactely the same
as a one-sample test of the differences against the value 0.

With kind regards

K.
 
#9
Sorry for the confusion. I'll try to rephrase it:

Let's start with more background...

In my task subjects have to infer the mean of a distribution which sometimes systematically changes. On every trial this mean + noise is presented (i.e on the first three trials the mean is stable and subsequently changes):


209
214
174
26
42
45
25
267
272
257
267

The difference between the two conditions is the frequency with which this mean changes. What I want to test is whether the prediction error is different between the two conditions (I would expect a higher averaged prediction error in the frequently changing condition).

My problem is that the magnitude of the prediction error does heavily depend on the randomly generated outcomes, that is, whether the mean changed or not.
To me it is not clear why, for example, trial 3 of the first condition is paired with trial 3 of the second condition as they do not have much in common (only that both have trial number 3).

I hope it's clearer now.
 
#10
No, you just have one subject who produces several responses.
Seemingly, there's no reason to perform a paired test, since
responses are not paired. I don't know how much sense it
makes to perform a significance test here (what do you want
to generealize to?), but independent samples tests look
appropriate.

With kind regards

K.
Ok thanks. At the moment I am just playing around with some parameters and want to check whether there are condition differences.

In the long-term I want to compare more subjects. Is it then necessary to run a paired t-test? If so, do I first compute the mean of every subject per condition and compare these?