Rank order data analysis

#1
I need the appropiate statistic tool to solve the following case:

I have done several experiments to 15 subjects.

Each subject rank 6 samples according to their personal opinion about an specific attribute.

They were allowed to repeat the marks given to the samples in case the subjects considered two or more samples at the same level.

So I have 15 rank data, from 1 to 6 or from 1 to 5,4,3.. (In that last situation, means that some marks are repeated).

Now I would like to analyse these results, in order to know if the attribute can be considered significant or, on the contrary, there is so variability in its rank marks that I could consider that there is no agreement between the subjects.

Thanks in anticipation
 

JohnM

TS Contributor
#2
You could to a Kruskal-Wallis test, which is the nonparametric version of the 1-way ANOVA. This would tell you if any one (or more) of the six samples has a median rank that is significantly different from the rest.

The hypotheses would be:

Null hypothesis - Ho: median ranks for all six samples are the "same"
Alternative hypothesis - Ha: at least one of the median ranks is different

A good explanation can be found here:
http://faculty.vassar.edu/lowry/ch14a.html

Often this test is done with "ratings" data that is transformed into ranked data, but you already have ranked data.
 
#3
Thanks a lot for the information.
Just one doubt. Taking for example, two subject that have ranked the 6 samples as follow:

subject A: 1 - 4 - 3 - 2 - 5 - 6
subject B: 2 - 3 - 2 - 1 - 4 - 5

Do I have to change the subject B's marks into the same scale as the subject A?. I mean, to change them into a 1 to 6 ranking scale?.
Because the sample number 6, althought has obtained different marks (6 and 5) it still means that has been chosen as the hightest one (I mean, if the second subject had used a 1 to 6 scale, the last sample would have obtained a 6 mark again, so it would change the average)

Thanks again for your help
 

JohnM

TS Contributor
#4
There are valid arguments for changing it and for not changing it.

In these cases, I usually advise the researcher to do the analysis "both ways" to see if it makes a difference in your conclusions.