Categorical data analysis (likert scale)

#1
Hello everyone, I'm working with categorical data on likert scale (4 scales) referred to the same sample of people (70 obs.) at Time0 and Time1. I have, for exemple, 9 different questions on a particular theme at the two different times.
It is the first time I work with this kind of data. I have already computed the frenquencies and represented them in a bar graph, someone could suggest me some index to calculate in order to show the differences between the ditributions in the two periods?
If I verify that my data are normally distributed can I use a paired t-test? It is statistically correct given that data are not continuos?
Some other ideas?

Thank you in advance.
 

Karabiner

TS Contributor
#2
So you have 4 Likert scales (i.e. 4 scales consisting of several Likert items), measured at
2 time points? What is the topic of your study, what are your research questions?
If I verify that my data are normally distributed can I use a paired t-test?
Normality of data is never necessary. At times, it might be useful to consider the prediction
errors/residuals of the model. For example, for the independent samples t-test, one could
look at the respective distribution of the dependent variable within each group. For the
dependent samples t-test you would consider the differences, not the variables themselves.
But all this is only needed in case of small samples (n < 30 or so). For larger samples like yours,
procedures such as t-test, ANOVA or linear regression are considered robust against non-
normality of the residuals.
It is statistically correct given that data are not continuos?
If they are Likert scales, as described above, then ususally they are used as interval scaled
measurements. You could display them graphically first, e.g. using box-and-whisker plots
for each scale at t0 and t1. If you want to analyse each scale separately, and you want to
know ehther its mean had changed significantly between t0 and t1, then dependent samples
t-test would be appropriate.

With kind regards

Karabiner
 
#3
Thanks for the reply.
I have to analyze a self-assessment questionnaire on the same group of people carried out at two different times and I have to understand how the answers have changed (the intent would be to demonstrate that the self-assessments after participation in the project have improved). At the moment the observations are 46 but they should increase.
For example, I have a group of 9 questions on a specific topic that can take values from 1 to 4 (1 a lot, 2 enough, 3 a little, 4 not at all).
I tried to do the box plots for the first 3 questions at time T0 and T1 but in some cases they don't come out well, for example for Q1 (see the attachment 1).
I was asked to find an index that summarizes and demonstrates the change that has taken place.
1. I had thought about using the paired t-test for every single question but I don't think it works...I tried for the first 3 questions but the p-values is always > 0.05 (see the attachment 2)
2. Could I also do the paired t-test on the mean calculated for each observation on the 9 questions at times T0 and T1?
3. Could you suggest me some other synthetic index to use on the whole set of 9 questions to show the differences in the answers between T0 and T1?

I hope to have been clear!
Thanks in advance
 

Attachments

Karabiner

TS Contributor
#4
Thanks for the reply.
I have to analyze a self-assessment questionnaire on the same group of people carried out at two different times and I have to understand how the answers have changed (the intent would be to demonstrate that the self-assessments after participation in the project have improved). At the moment the observations are 46 but they should increase.
For example, I have a group of 9 questions on a specific topic that can take values from 1 to 4 (1 a lot, 2 enough, 3 a little, 4 not at all).
Please try to be more precise with regard to the description of your measurement(s) and the
research goals. You write something about single items (how many in total, beyond the "for example 9"), and
some of them seem to be "on a specific topic". And you said something about "4 scales" -- or did
you not mean 4 scales, but item response formats with 4 values?
I was asked to find an index that summarizes and demonstrates the change that has taken place.
You were asked by who? And what is the topic of the research, what are the research questions,
how many items do you have in total, where did the items come from, how can you tell which
one belong together? Have the items been used in earlier studies and/or do you use well-known
instruments?

What I can say is that you should probaly not use t-tests with 4-point scales, rather you should
consider the sign test.

With kind regards

Karabiner
 

Buckeye

Active Member
#5
I did a very similar analysis for my thesis. I analyzed a survey with 6 different groups of questions each measuring a different aspect of "attitude". The questions were on a Likert scale from 1 to 7. I wanted to analyze the difference in attitude pre versus post intervention compared to a control group. So, I added the scores together within each group of questions to comprise a total score. Then, I took the difference of this score = post score - pre score. Finally, I used Hotelling's T test to measure if there is a difference in at least one attitude component between treatment and control groups.
 
#6
The analysis of Buckeye is similar to mine.
In order to write a monitoring report on a project, I have to analyse a survey with 6 different groups of questions each measuring a different ability (in each group I have from 6 to 9 questions). The questions are on a Likert scale from 1 to 4 (1 a lot, 2 enough, 3 a little, 4 not at all).
I would like to evaluate, for each group of questions, if these abilities have increased from time 0 (when the project started) to time 1 (after 1 year).
Karabiner at the end of his last post writes that with a 4-point scale I can't use T-test (could I ask you why?), so I suppose I can't probably use Hotelling's T test too?
About the sign test, it makes sense to calculate the median or the mean or the total for each observation within each group of questions at time 0 and time 1 and then use the sign test to compare these 2 new variables?

Thanks a lot.
 

Karabiner

TS Contributor
#7
In order to write a monitoring report on a project, I have to analyse a survey with 6 different groups of questions each measuring a different ability (in each group I have from 6 to 9 questions). The questions are on a Likert scale from 1 to 4 (1 a lot, 2 enough, 3 a little, 4 not at all).
Such a 4-point scale is not called a Likert scale. Likert scale is the name of a measurement
instrument consisting of several Likert-type items, jointly measuring a construct.
If the term "Likert scale" popped up during the preparation of your study, then either it was
simply used incorrectly (as it happens in about 87% of studies), or it was actually meant to
express that the 6 to 9 items measuring some ability togehter form a Likert scale. In the latter
case, summing up the scores of the 6 to 9 items would be the ususal practice.
But you did tell nothing about where the items and the item groups came from, whether they
were simply made up, or maybe taken from literature, or something else. Therefore it is
not easy to know what to assume.
Karabiner at the end of his last post writes that with a 4-point scale I can't use T-test (could I ask you why?),
A 4-point rating scale is usually considered as having the ordinal level of measurement.
For t-test, you need an interval scale. But I would rather not recommend to perform
about 40 to 50 (?) pre-post comparisons with single items, regardless of whether
the t-test is used or the sign test. If you construct a scale out of the 6-9 items measuring
an ability, then this resulting scale is usually considered interval, and you can use
t-tests on them.
About the sign test, it makes sense to calculate the median or the mean or the total for each observation within each group of questions at time 0 and time 1 and then use the sign test to compare these 2 new variables?
Well, that is not quite how the sign test works. Basically, it just looks at the
positive and negative changes between t0 and t1. Why not let the software
do that for you.

With kind regards

Karabiner
 
#8
About items and items gruops I don't know if they have been made up or taken from literature....I haven't prepared the survey, but I could ask!

A last question, when you talk about 'construct a scale out of the 6-9 items measuring an ability' what do you mean? How can I do?

Thank you a lot for your help!
 

Karabiner

TS Contributor
#9
Sorry, that was a sloppy formulation. If one may consider the 6-9 items of a group as a joint measurement
of an ability, then ususally one would sum them up.