Ordinal DV and Categorical IV - Correct test

#1
I have a self-efficacy scale (DV). my self-efficacy scale is a liker scale (ordinal; 1 to 7, strongly agree to Strongly disagree) I asked the same question in pre and post examination. I have 7 different sites. I would like to know if the change in self efficacy varies across sites. What would be the correct test for this situation?

Thank you,
Marvin
 
#2
Normally I would say repeated measure ANOVA, but that requires an interval DV. One possibility is to make your DV the change in efficacy (responses would range from 0 to 6) and do ordered logistic regression with dummies corresponding to your sites. This would tell you if the change in any given site is signficantly different than a reference group (or the mean of the means depending on how you run your data - the former is much more common in logistic analysis).

If all you want to do is see if there is a difference in site you can just generate a median value on the change in the DV by site (or mean if you assume the distance between levels is the same which many do with likert data). This won't generate a statistical test.
 

Karabiner

TS Contributor
#3
Personally, I would treat a strongly agree - strongly disagree
scaled item with 7 levels as close enough to interval scale, so
noetsis suggestion of repeated measures ANOVA would apply.
Depends on your reviewer, though, whether this is accepted. And
it may be a matter of sample size (which was, as surprisingly
usual in this forum, not mentioned).

With kind regards

K.
 
#4
Thank you both for you valuable suggestion. But I am kind of new in quantitative analysis. So I have seal-efficacy scores in pre and post for all the centers. The first thing I have to do is to generate a variable that measure the change in each center. My plan was to do gen change_efficacy = efficacy_post - efficacy_post. And then do a test or regression. Is this valid. Please can you walk me thru this?

What are the commands for your comments above. I am generating a report for my job and need this info. I want to perform a valid analysis.

Thank you very much.

Marvin
 
#5
You chose a difficult exercise for someone new to quantitative analysis.:) What you suggested will work. Note that contrary to my previous comment you will have 12 levels of your dependent variable (because you will have some cases where efficacy will go down so possible values will be -6 -5........6 7). Given that just do repeated measure ANOVA as Karabiner suggested.

Unfortunately I can't help with commands - I work in SAS.
 
#6
Hi Noetsi.. thanks for your response.. heheeh yes I know is difficult but I have no other choice ...
yOne possibility is to make your DV the change in efficacy (responses would range from 0 to 6) and do ordered logistic regression with dummies corresponding to your sites". How can I do this? change_efficacy = efficacy_post - efficacy_post .. but I have negative number and I believe stata doesnt work with negative numbers in regression.

"If all you want to do is see if there is a difference in site you can just generate a median value on the change in the DV by site " how can I do this?

"Given that just do repeated measure ANOVA as Karabiner suggested." So how woulr this work.. dont tell me the commands but the logic.


Thank you again..

I greatly appreciate...
 
#7
First you would subtract the before from the latter and create a new variable. This should give at most 12 distinct values (from negative 6 to positive six). You have to make a decision at this point if you are only interested in how much change occured (in which case you want to take the absolute value of the numbers above leaving you with six distinct levels from 1 to 6) or if direction matters as well in which case you have 12. Direction here would reflect that in some cases efficacy may go down from before to after (for instance from 5 before to 4 later) and in some cases it will go up, for example from 4 before to 5 later.

In the first case you will get negative numbers and in the second case you will get positive ones. But the point is you have to decide if how much change is all you care about or direction as well - you will end up with 6 distinct levels with how much and 12 distinct levels if you use direction. (Note this is the theoretical number. In practice someone might not actually be at this level of change, you should look at how many distinct levels are actually in the results).

You have to convert your sites to dummy variables, for each variable it will be 1 if the case is in that site and 0 if not. One site is not coded (I believe, this is the case for repeated measure ANOVA, it is what regression does, but you should look this up in a discussion of repeated measure ANOVA to be sure) These are your predictors. If you end up with 9-12 distinct levels you just run repeated measure ANOVA in Strata at this point. I don't work with that method enough to advice you on it, you will have to get a text or look on line for help there (or hopefull another poster with more expertise in it than I will wander by).

If you end up with 6 levels (when only how much change is the way you went) it is iffy to use repeated measure ANOVA. Authors disagree on the impact of this (technically the DV should be interval with many levels). You might instead try ordered logistic regression. Regardless of which you do you need to test for the assumptions of the method. I suggest getting a text like Tbachnick and Fidel "Using Multivariate Statistics" The issues are too varied to address here.

A simpler approach might be to simply use chi square (with your predictor being a single variable with one level for each of the sites). You can also generate a simple summary of the average change per site to physically look at the differences in sites (technically you use the differences in the median value although it is common to ignore this).
 
#8
Thank you noetsi... but I still have some questions..
If i generate a variable with the change in self-efficacy do I still need to do repeated measure ANOVA? Cant i just do a One way ANOVA

one effmedianchange siteid, tab

1. There are some caregories in my categorical variable (sites) that have only =<8 observation. Would this affcet my analysis?
2. to generate the variable effmedianchange, I subtract efficacy_post-efficay_pre. But there are ordinal variables. Is this allow?

Also I was reading and it seems that I have to reshape my data into wide format in order to do repeated measure ANOVA. If I do reapeted measure anova do I still have to create the effmedianchange variable?

Thank you. I would appreciate if any of the Stata guys help me with this.

Marvin