Nonparametrics Help

[bjackson55]

I work with non-human animals. So I am almost always limited to using nonparametrics, however my university does not teach it.
For my thesis, I am have 4 subjects. They will be completing 3 tasks, Relevant1, Irrelevant, and Relevant2. Criterion is that they must achieve .85 accuracy over 120 trials to move to the next task. I want to perform two tests. One to see if there is any significant difference between the relevant and irrelevant conditions as far as overal accuracy. Another to see if there is a difference in speed, how quickly (in trial count) they finish each condition. I was originally pointed in the direction of the McNemar test, however I am not sure if this is the best test as I believe it requires approx 20 subjects, and I only have 4. Any help would be much appreciated.

Cheers!

 

[Dason]

I work with non-human animals. So I am almost always limited to using nonparametrics,

I'm not entirely sure how the first point implies the second

 

[bjackson55]

I'm not entirely sure how the first point implies the second

Often when you work with animal populations you do not have a large enough sample size. My largest sample size has been 10. So I am almost never able to use typical regression models or ANOVA.

 

[Dason]

I still don't quite agree with that assessment. Small sample sizes are when it makes the most sense to make the assumptions if it's reasonable. It might be a reasonable thing to do in your field but I just don't see the immediate connection between your original first point and the conclusion.

 

[bjackson55]

I still don't quite agree with that assessment. Small sample sizes are when it makes the most sense to make the assumptions if it's reasonable. It might be a reasonable thing to do in your field but I just don't see the immediate connection between your original first point and the conclusion.

That's fine to disagree. But do you have a solution to my problem? A potential test that may work?

 

[Karabiner]

Dason (if I understand him correctly) is correct. If you do not have strong assumptions about the distribution
of your variables in the population, what's the use of trying to perform statistical tests of signficance with
just n=4?

With kind regards

Karabiner

 

[bjackson55]

Dason (if I understand him correctly) is correct. If you do not have strong assumptions about the distribution
of your variables in the population, what's the use of trying to perform statistical tests of signficance with
just n=4?

With kind regards

Karabiner

Hello. As mentioned above, in animal testing (especially non human primate testing) you never have a n > 10. The animal literature, which is quite extensive, still uses statistical testing. So it is obviously possible. Being a young student in the program I just am not sure how.

 

[fed2]

especially non human primate testing

I totally sympathize, if you want to see a fist fight among scientists, start talking about increasing nhp numbers! Im supprised you get anywhere close to 10.

Small sample sizes are when it makes the most sense to make the assumptions if it's reasonable.

Im glad to hear someone say that, since ive felt the same secretly for a while now. On the other hand i have noted some tendancy to use non-parametric tests on nhp data, but I guess it depends on the field. I'm not really sure why that is.

 

[bjackson55]

I totally sympathize, if you want to see a fist fight among scientists, start talking about increasing nhp numbers! Im supprised you get anywhere close to 10.


Im glad to hear someone say that, since ive felt the same secretly for a while now. On the other hand i have noted some tendancy to use non-parametric tests on nhp data, but I guess it depends on the field. I'm not really sure why that is.

In the stats classes I’ve taken in my program (cognitive science) I’ve always been taught that if you have less than 30 participants you can’t run a regression or ANOVA. So you must use nonparametrics instead. You’re also very rarely going to meet assumptions when using NHP populations.

 

[fed2]

I’ve always been taught that if you have less than 30 participants you can’t run a regression or ANOVA.

I guess you got your rules to follow. There might be reasons for that in this case i am not aware of. Generally though, Id say i don't really see it. Its pretty routine to run parametric tests on immunological outcomes with n smaller than that.

You’re also very rarely going to meet assumptions when using NHP populations.

Yeah its the cunnundrum of small sample sizes. Do you run a non-parametric test, which ends up requiring near complete separation between treatment and control ranks, and has like nill power, or do you make some ambitious assumptions that aid the inference. Theres some pretty solid arguments for the latter, to spite what your text-book says. Well welcome to the world of NHP studies, its filled with compromises, contradictions, and a little hypocrisy. Your professor is probably running a t-test as we speak on n=5.

 

[bjackson55]

Unfortunately my

I guess you got your rules to follow. There might be reasons for that in this case i am not aware of. Generally though, Id say i don't really see it. Its pretty routine to run parametric tests on immunological outcomes with n smaller than that.


Yeah its the cunnundrum of small sample sizes. Do you run a non-parametric test, which ends up requiring near complete separation between treatment and control ranks, and has like nill power, or do you make some ambitious assumptions that aid the inference. Theres some pretty solid arguments for the latter, to spite what your text-book says. Well welcome to the world of NHP studies, its filled with compromises, contradictions, and a little hypocrisy. Your professor is probably running a t-test as we speak on n=5.

Unfortunately my professors only work with clinical data and so they can’t help me. Hence why I’ve turned to this forum for help. Which, doesn’t seem to be helping either, as no one has offered a nonparametric test that would fit my needs.

 

[fed2]

Which, doesn’t seem to be helping either,

yeah i sort of missed the mark on that, but i guess the forum acts in mysterious ways.

 

[Karabiner]

They will be completing 3 tasks, Relevant1, Irrelevant, and Relevant2. Criterion is that they must achieve .85 accuracy over 120 trials to move to the next task.

So, necessarily all participants must have achieved > 0.85 accuracy in Relevant1 and in Irrelevant,
otherwise they would not have moved on to the next task?

 

[katxt]

I think we are all interested in helping. Perhaps it would help our understanding if you made up a mock data set and posted it here.. We could try various things.