Statistical test for ranked data among 3 groups - ANOVA or Kruskal-Wallis?

#1
Hi all,

I've read several books but I am not sure which statistical test to apply and whether my thinking is correct so I would greatly appreciate any help.

I conducted a survey among three independent groups (managers N=110, analysts N=105 and investors N=80) and in one question I asked them to rank, based on their importance, the 5 out of 14 available financial metrics used when analyzing a company (with 1 being the most important down to 5 and leaving the other metrics blank). I then constructed scores for the responses (a rank 1 will get 5, the rank 2 will get 4 etc. and no rank 0) to establish/order which metrics are on average more important overall and for each group of respondents. Now I would like to compare the three groups for each of the 14 metrics and I am not sure which test (I use SPSS) to use and these are my thoughts.

1) I tried to use an ANOVA but since a rank could be considered an ordinal variable I am not sure it is still appropriate? As only for 4 metrics out of 14 the Levene's test was not significant, I applied the Welch Test in case of heterogeneous variances. The ANOVA showed that there were statistical differences at the p<0.05 level among the three groups for 6 out of the 14 financial metrics under study. Post-hoc (both Scheffe and Games-Howell given the unequal sample sizes and unequal variances instances) analysis indicated that there was a significant difference in means between some groups for some metrics. Is this approach correct?

2) Given the issues mentioned in point 1, I also ran the Kruskal-Wallis tests + post-hoc Dunns-Bonferroni pairwise comparisons (in the SPSS model viewer), which led me to essentially the same results. However, I am not sure that this is the right test either given the heterogeneous variances.

Which of the two options is the better one (or is there a 3rd better one)? Shall I state in my dissertation that I ran both tests?

Many thanks in advance for any help.
 
#3
Thanks a lot. Would you still apply the same reasoning if the analysis were run on questions with the same respondents as above that had a Likert scale as answer choices or would in such a case an ANOVA be more appropriate?
 

Karabiner

TS Contributor
#4
It depends on whether it really is a Likert scale (composed of several items
which are summed up), or just a single Likert-type item with a special answering
format (which very often is falsely called a Likert scale). Only the first is usually
treated as interval scaled, the latter as ordinal.

With kind regards

Karabiner
 
#5
Thanks a lot for the quick turnaround.
I assume that Likert types of answers
Extremely likely / Somewhat likely / Neither likely nor unlikely / Somewhat unlikely / Extremely unlikely
Strongly agree / Somewhat agree / Neither agree nor disagree / Somewhat disagree / Strongly disagree
Extremely important / Very important /Moderately important / Slightly important /Not at all important
which are then re-coded on a scale from 1 to 5 should be considered as "false Likert scale" and as such treated as ordinal and therefore analyzed with the KW Test?
 

Karabiner

TS Contributor
#6
There is no such thing as "false Likert scale". Likert scales is a term for an isntrument which consists of several Likert-type items.
The response scale of a single Likert-type item (ordinal) is not called Likert scale (treated as interval).

With kind regards

Karabiner