Wilcoxon Signed Rank Test for Small Sample Size

#1
Hello!

I am a bit stumped with the research I am conducting and wanted to ask for some advice/insight.
My study involved having 4 individuals complete the same 3 measures at pre- and post-test. So each individual completed each measure before receiving an intervention, and then completed the same measures again after receiving the intervention. Ordinarily, this would be a t-test looking at pre/post differences across individuals. However, due to the small sample size, the normality of my data is called into question. I decided to use a Wilcoxon Signed-Rank test to compute the Z statistic and p-value.
Well....for some reason, I keep getting the exact same Z statistic and p-value, regardless of the data I am putting in (for example, when I compare the pre/post scores on measure A, I get the same exact values as when I compare the pre/post scores on measure B). Obviously, this is incorrect.
I read online that the very small sample size (this is a multiple baseline study) can have an impact on the Wilcoxon-Signed Rank test and produce the weird trend described above. My questions are:

1) Is this actually true? Is this considered "normal" when running this type of analysis with a small sample?
2) If it is, what should my next steps be? I am hesitant to just drop this into my paper and call it a day...it doesn't seem very accurate or meaningful...

Note: I also have adjusted my level of significance from 0.05 to 0.10 due to the small sample size. Not sure how relevant that info is, but wanted to include it.

Any help is greatly appreciated!!
 

Karabiner

TS Contributor
#2
With just 4 pre-post values, it is quite possible that the pattern of results (rank sum for the
positive differences, and rank sum for negative differences) is the same between analyses.

And the calculation of approximate z value uses the sample size (here, always n=4), and one
of the two rank sums. But it is not a useful test statistic for sample sizes < 20, AFAIR.

With kind regards

Karabiner