Stats Novice: Appropriate statistical test for categorical dependent variable with unpaired control/treatment IV?

Cpriz

New Member
#1
Hello!
I am very new to statistics and am trying to figure out the appropriate statistical test to use for a small study.
We only have 38 participants. Half are in the control group (placebo), half are in the treatment group, and the groups are unpaired. Both groups have severity score outcomes assessed pre- and post- treatment/placebo. The severity score outcome is initially on a continuous scale, but these numbers are placed into functional groups (no disease, mild, moderate, severe, very severe). 0 is no disease, otherwise, the categories are separated at regular numeric intervals. The outcomes are not normally distributed, with most of the participants falling in the categories of no disease or mild disease. When looking at the raw data, all of the scores only changed by 1 category or didn't change at all, so the change scores are all either -1 (worse), 0 (no change), 1 (better).

The primary research question is what is the effect of treatment on severity score.
I think one way to do this would be to compare the change scores for the controls versus treatment groups to see if there is a significant difference. I think this would result in 2 independent variables (placebo, treatment) and 3 dependent variables (worse, no change, better). From my reading it seems like the sample size is too small for Chi square, so perhaps I should use Fisher's exact test? I am also wondering if the Wilcoxon-Mann Whitney test is appropriate for this? Or is there another more appropriate test that I haven't considered?

Please help direct me if I'm thinking about this the right way and what would be the most appropriate statistical test. I will be performing the analysis in Stata.

Thank you!
 

Karabiner

TS Contributor
#4
The severity score outcome is initially on a continuous scale, but these numbers are placed into functional groups (no disease, mild, moderate, severe, very severe).
This might make practical sense or not, but is it necessary to use these groupings for the
statistical analyses of the study? Your research question is what is the effect of treatment
on severity score
? So I'd suggest that you use the initial score, perform a mixed analysis
of variance (i.e. 1 group factor and 1 repeated-measures factor), and use the grouping
for descriptive purposes.
If this is too complicated, then just calculate the pre-post difference for each subject
ad compare the difference scores between groups using t-test.
3 dependent variables (worse, no change, better).
It is 1 dependent variable with 3 possible values. But collapsing all changes into just 3
possible outcome categories would waste information about the magnitude of change.
From my reading it seems like the sample size is too small for Chi square,
n=38 would be a typical sample size for Chi², but your outcome is ordinal. This
would rather be compared between groups using Mann-Whitney U-test, as
you already considered.

With kind regards

Karabiner
 

hlsmith

Less is more. Stay pure. Stay poor.
#5
Was randomization deemed successful? Good balance between groups. Was follow-up also the same between subjects and groups?
 

Cpriz

New Member
#7
This might make practical sense or not, but is it necessary to use these groupings for the
statistical analyses of the study? Your research question is what is the effect of treatment
on severity score
? So I'd suggest that you use the initial score, perform a mixed analysis
of variance (i.e. 1 group factor and 1 repeated-measures factor), and use the grouping
for descriptive purposes.
If this is too complicated, then just calculate the pre-post difference for each subject
ad compare the difference scores between groups using t-test.

It is 1 dependent variable with 3 possible values. But collapsing all changes into just 3
possible outcome categories would waste information about the magnitude of change.

n=38 would be a typical sample size for Chi², but your outcome is ordinal. This
would rather be compared between groups using Mann-Whitney U-test, as
you already considered.

With kind regards

Karabiner
Thank you for your response!
I see what you mean... I think it might actually simplify things to use the mixed ANOVA and assign the groupings after analysis for descriptive purposes. Would there be any issue with this test if there is a skew of participants with no disease, ie a score of 0? Is there anything I have to do to adjust for that?
Do you think the mixed ANOVA would be a better test than the Mann-Whitney U-test because it would provide more information about the magnitude of change?