Repeated measures with varying count between group

#1
I'll explain my problem a bit:
I have two groups S(story) and P(picture).
Each is composed of the same 48 subjects.
My endpoint is a 'score' on an ordinal 4 point scale.

The 48 subjects create varying number of stories. Each story is graded on that four point scale. Some subjects made 2 stories. Some made 6. The stories were split between 7 reviewers and each had to only grade 1 (this is time intensive). So each subject has between 2 and 6 scores.

The 48 subjects also create one picture. This picture is graded on the same scale and can be done quickly. All 7 reviewers grade each picture. So each subject has seven scores in this group.

Now my question is how on earth do I compare the scores from S to P.

Is it some type of 2-way repeated measures nonparametric test with a lot of missing data?
I'm really lost on what I should do and how I can achieve it in SPSS.
Thanks for any help!
 
#2
One quick way is to get the average picture and average story score for each subject (48 pairs of scores), and see if there is a significant correlation between S and P.
 
#3
Thank you for the suggestion. Will reviewers harshly criticize transforming an ordinal variable into a continuous variable in order to do averages? I was hoping to be able to analyze in terms of proportions of "average" vs "exceptional", etc. (in terms of the ordinal scores).

One quick way is to get the average picture and average story score for each subject (48 pairs of scores), and see if there is a significant correlation between S and P.
 
#4
Will reviewers harshly criticize transforming an ordinal variable into a continuous variable in order to do averages? I was hoping to be able to analyze in terms of proportions of "average" vs "exceptional", etc. (in terms of the ordinal scores).
Do you want to find out if subjects' picture scores are reflected in their story scores? That is, do high picture scorers tend to have high (or low perhaps) story scores?
Assuming this is what you are looking for, the correlation approach is fine. If you want a non parametric approach, you can use Spearman's correlation.
You can also do what you suggest, which is really a slightly weakened form of the correlation - put each subject into one "average" vs "exceptional" groups for each of picture and story, make a 2 way summary table and use Chi Square.
 
#5
I'm looking to see if the easy and quick way of reviewing a picture gives the same score as the time intensive review of a narrative essay. The hypothesis is that the scores will be the same between groups. But yes, if there was a difference (whether the picture gives a higher or lower score) would be interesting to find.

I've now computed the mean and median of the various scores for each subject and then did a nonparametric correlation of those values.

This was easy to do. I'm just curious how much 'robustness' I'm losing by turning the multiple scores into a single aggregate measure. I 'thought' I should be doing an analysis that considers the scores in a 'repeated measures' sense. But if condensing multiple scores into a single average value isn't statistical heresy than I'm happy :)

Thanks for taking the time to respond!


Do you want to find out if subjects' picture scores are reflected in their story scores? That is, do high picture scorers tend to have high (or low perhaps) story scores?
Assuming this is what you are looking for, the correlation approach is fine. If you want a non parametric approach, you can use Spearman's correlation.
You can also do what you suggest, which is really a slightly weakened form of the correlation - put each subject into one "average" vs "exceptional" groups for each of picture and story, make a 2 way summary table and use Chi Square.
 
#6
If your aim is to see if the scores on one test are significantly different from the other test, then you could use a two sample paired test either the t test or the Wilcoxon test. However you will still have to use the averaged data for each subject.