Hello,

I'm working as a volunteer on a patient led project to assess the effectiveness of a newly released medical device and hoping that I may get some help from the experts on the forum here, mainly to see if I am on the right track.

I have a mathematics background and did some stats at Uni but that was quite a while ago. To get myself up to speed I have been furiously reading stats books! I am using excel and Real stats (http://www.real-statistics.com/), which I am finding quite adequate.

Anyway, here is the problem I am addressing:

We have a number of people in our forum who have volunteered to provide feedback on the new device. We ask the users to rate the severity of the condition between 1 and 5 (Lets say Severity scale) and also (optionally) to answer a questionnaire which gives a severity result between 1 and 100 (lets say it is ABC scale). Most people are answering the optional bit (we made it optional as we thought we might otherwise have too many dropouts, but this has not proven to be the case and in retrospect we should probably have made it mandatory!). According to the literature, a change in 13 points or more on the ABC scale is supposed to be a significant change.

We've also asked a heap of questions about various factors, such as Age, the duration of the condition, Gender and quite a few others (about 12 in all). Each factor is divided into mostly 4 groups.

They answer the severity questions at 0,6 and 12 weeks of treatment with the new device, and the various factor questions at 0 weeks.

We will most likely, after drop outs, have data to 12 weeks for about 35 patients (which I know is not a great sample size!). We are about 50% of the way through at the moment.

These are the tests I am doing;

1. One factor ANOVA on the ABC scale deltas at 6 and 12 weeks (delta from 0 weeks) against each of the factors and then Tukey HSD with Bonferroni correction for contrasts (4 of 12 factors are significant to date).

2. One-tailed paired t-tests on the ABC scale at 6 and 12 weeks from 0 weeks. Both are significant (although 6-12 is not and is actually going in the wrong direction).

3. Wilcoxon Signed Rank paired test on the changes in the Severity Scale at 6 and 12 weeks from 0 weeks (again both are significant)

I am also keen to do tests which mitigate being unduly disrupted by outliers. For example, one patient has recorded a drop from about 95 to about 20 on the ABC scale. As such, I am also classifying patients into those that experienced more than a significant drop (13 points) and those that that did not. (i.e. a 1 or a 0). With this data I have used these stats:

1. Fisher Test to determine independence of the variable describing significance and the variable describing each factor. I have used this rather than the chi-squared test s the former supports smaller sample sizes (3 factors significant).

I would also like to use another test which will do the equivalent of the ANOVA, but for the significant/not significant dependent variable (rather than the ABC scale).

I originally investigated Logisitic Regression, which has the advantage that is can be used a predictor, but unfortunately the sample size has to be around 100.

The other test I thought about using was the Kruskal-Wallis + Dunn Test (as the significant/not significant variable is binary it is definitely not Normal). I've done one of these tests and getting reasonable results, but just wondering if this is the right thing to do?

EDIT: After further reading I have my doubts about using KW in this way. Not sure how I could focus on the patients who have achieved a significant change rather than the mean (which could be heavily influenced by one patient).

Anyway, if anyone can help with this project (the Kruskal-Wallis question and the approach in general), it would be much appreciated. It's for a good cause, the condition can be debilitating, and it's run by patients rather than medical companies!

I'm working as a volunteer on a patient led project to assess the effectiveness of a newly released medical device and hoping that I may get some help from the experts on the forum here, mainly to see if I am on the right track.

I have a mathematics background and did some stats at Uni but that was quite a while ago. To get myself up to speed I have been furiously reading stats books! I am using excel and Real stats (http://www.real-statistics.com/), which I am finding quite adequate.

Anyway, here is the problem I am addressing:

We have a number of people in our forum who have volunteered to provide feedback on the new device. We ask the users to rate the severity of the condition between 1 and 5 (Lets say Severity scale) and also (optionally) to answer a questionnaire which gives a severity result between 1 and 100 (lets say it is ABC scale). Most people are answering the optional bit (we made it optional as we thought we might otherwise have too many dropouts, but this has not proven to be the case and in retrospect we should probably have made it mandatory!). According to the literature, a change in 13 points or more on the ABC scale is supposed to be a significant change.

We've also asked a heap of questions about various factors, such as Age, the duration of the condition, Gender and quite a few others (about 12 in all). Each factor is divided into mostly 4 groups.

They answer the severity questions at 0,6 and 12 weeks of treatment with the new device, and the various factor questions at 0 weeks.

We will most likely, after drop outs, have data to 12 weeks for about 35 patients (which I know is not a great sample size!). We are about 50% of the way through at the moment.

These are the tests I am doing;

1. One factor ANOVA on the ABC scale deltas at 6 and 12 weeks (delta from 0 weeks) against each of the factors and then Tukey HSD with Bonferroni correction for contrasts (4 of 12 factors are significant to date).

2. One-tailed paired t-tests on the ABC scale at 6 and 12 weeks from 0 weeks. Both are significant (although 6-12 is not and is actually going in the wrong direction).

3. Wilcoxon Signed Rank paired test on the changes in the Severity Scale at 6 and 12 weeks from 0 weeks (again both are significant)

I am also keen to do tests which mitigate being unduly disrupted by outliers. For example, one patient has recorded a drop from about 95 to about 20 on the ABC scale. As such, I am also classifying patients into those that experienced more than a significant drop (13 points) and those that that did not. (i.e. a 1 or a 0). With this data I have used these stats:

1. Fisher Test to determine independence of the variable describing significance and the variable describing each factor. I have used this rather than the chi-squared test s the former supports smaller sample sizes (3 factors significant).

I would also like to use another test which will do the equivalent of the ANOVA, but for the significant/not significant dependent variable (rather than the ABC scale).

I originally investigated Logisitic Regression, which has the advantage that is can be used a predictor, but unfortunately the sample size has to be around 100.

The other test I thought about using was the Kruskal-Wallis + Dunn Test (as the significant/not significant variable is binary it is definitely not Normal). I've done one of these tests and getting reasonable results, but just wondering if this is the right thing to do?

EDIT: After further reading I have my doubts about using KW in this way. Not sure how I could focus on the patients who have achieved a significant change rather than the mean (which could be heavily influenced by one patient).

Anyway, if anyone can help with this project (the Kruskal-Wallis question and the approach in general), it would be much appreciated. It's for a good cause, the condition can be debilitating, and it's run by patients rather than medical companies!

Last edited: