Unpaired repeated measures test?

#1
My dataset is pre- and post-test Likert Scale around an intervention, but it is not matched: it is collected anonymously from the same cohort of students (therefore I assume some small change in who's in class on each testing day).

I am struggling with the analysis of this data. I can group responses from the scales and compare proportions pre- and post-, but haven't figured out the best way to compare full responses. Chi-square, Wolcoxon, t-tests, etc. all require, as I understand it, independent groups, which repeated measures are not.

I am also not clear on how to approach the idea of summing several responses and running ANOVAs, or if that's even possible.

It's amazing how much information is online about how to do all these tests, but there appears to be a dearth of information about the details of selecting appropriate tests!

Thank you all for thoughts and feedback,

Peter
 
#2
You could just perform a paired t-test (or equivalently one sample t-test on the change scores (post-pre)) to see if the mean change is significantly different from 0. If a parametric test isn't appropriate (if you have a small sample size and data are far from normal) you could use the Wilcoxon test instead.
 
#3
You could just perform a paired t-test (or equivalently one sample t-test on the change scores (post-pre)) to see if the mean change is significantly different from 0. If a parametric test isn't appropriate (if you have a small sample size and data are far from normal) you could use the Wilcoxon test instead.
But I have been told that a paired t-test is only valid if the two groups are independent. Both my groups are the same people, simply tested at different times, so they're not independent. Am I misunderstanding?

Thanks for your thoughts!
 

Dason

Ambassador to the humans
#4
Well you probably misunderstood whoever told you that because the only time a paired t-test is appropriate is if there is some sort of correlation between the observations (as in the case where you have two observations on the same person).
 
#5
I have been struggling with the same question. I have "repeated" data, in that I have repeated measures on the same group. However, I cannot "pair" the data due to the nature of the test (like the anonymous scores here. I'm not sure which score goes with which individual). Would an ANOVA (I have more than two time points) with a repeated measure correction be appropriate?
 

Jake

Cookie Scientist
#6
No. You cannot do a repeated measures analysis if you don't know which measurements are the repeated ones for each unit. I'm afraid the best you can do is to pretend that your pre- and post-scores are independent and do a regular t-test. Unfortunately this test will be biased in the direction of being overly conservative.
 

staassis

Active Member
#8
If there is some, unknown overlap between the pre- and post- groups but the subjects of the groups are not matched, then BOTH independent samples t-test and paired t-test are not valid... However, there is a way out. Use PROPENSITY SCORE MATCHING based on many other variables in the data set to match a subset of subjects in the pre- group to a subset of the subjects in the post- group. Then run paired t-test on the two matched subsets.