I have performed an experiment where I tested how long it took users to reach various points on table and their perception of how difficult the task was to perform under various conditions. The user used either a handheld robot arm, or a handheld wand to reach the required points. In addition, the user was given information about how to reach the points by one of four feedback methods: a handheld LCD screen, a see-through augmented reality headset, a virtual reality headset, or gesturing actions performed by the robotic arm.
Gesturing feedback with the wand was not technically possible because it did not have motors in it.
I used a repeated measures design where 21 participants were tested once on each of the 7 modes. Note that we cannot test the (Wand - Gesturing) condition and it is left blank
I measured the results using 1-way repeated measures anova and used the Bonferroni correction on the posthoc analysis to produce a table of pairwise comparisons.
I submitted this as a paper but one of the reviewers said I should use 2-way repeated measures ANOVA instead of 1-way repeated measures.
In some ways I can see why they are suggesting 2-way ANOVA because I have two independant variables, device (robot,wand) and feedback (HHD,VR,AR,Gesturing). However, I was not able to record the last condition (wand-Gesturing) and my statistics software (SPSS) will not allow me to perform 2-way ANOVA without the missing data.
I have done a 2-way repeated measures ANOVA on the first 3 columns of the table excluding the gesturing condition, but this is unsatisfactory because I want to compare the Robot-Gesturing to the other Wand feedback methods.
I have spoken to people in my office who suggested testing for a difference between the Robot and Wand condition first, then testing for differences in feedback methods for the Robot condition and then for the wand condition independently. This has the same problem where I can't compare the Gesturing feedback to any of the Wand conditions.
What I would like to do is say that I am not considering the device (Robot/wand) as an independent variable, but instead consider each of the 7 modes as independent. In reality there is a correlation in performance between the Robot/wand so I don't think this is valid. However, is there a correction I perform that takes this correlation into account so that I can use 1-way repeated measures ANOVA and still perform pairwise comparisons between all the different conditions?
Thanks for you attention.
Appendix:
The paper is over the attachment limit so this is a dropbox link: https://www.dropbox.com/s/0xmebzara9dacrq/FeedbackPostReview.pdf?dl=0
An extract from the reviewer's comments:
"This study has 2 independent variables: 1) the device and 2) the feedback mode (this is called operation mode in Section III, which I believe is poorly named). Hence, the experimental design is 2-way. However, the analyses conducted in the paper utilize a 1-way ANOVA. This is technically incorrect. The authors need to report 2-way ANOVA results, which would clearly and reparately indicate whether there is a significant effect of feedback method and/or the device choice."
Gesturing feedback with the wand was not technically possible because it did not have motors in it.
I used a repeated measures design where 21 participants were tested once on each of the 7 modes. Note that we cannot test the (Wand - Gesturing) condition and it is left blank
Code:
╔═══════╦═════════════════╦═════════════════╦═══════════════════╦═══════════╗
║ ║ Handheld Display║ Virtual Reality ║ Augmented Reality ║ Gesturing ║
╠═══════╬═════════════════╬═════════════════╬═══════════════════╬═══════════╣
║ Robot ║ [] ║ [] ║ [] ║ [] ║
║ Wand ║ [] ║ [] ║ [] ║ ║
╚═══════╩═════════════════╩═════════════════╩═══════════════════╩═══════════╝
I submitted this as a paper but one of the reviewers said I should use 2-way repeated measures ANOVA instead of 1-way repeated measures.
In some ways I can see why they are suggesting 2-way ANOVA because I have two independant variables, device (robot,wand) and feedback (HHD,VR,AR,Gesturing). However, I was not able to record the last condition (wand-Gesturing) and my statistics software (SPSS) will not allow me to perform 2-way ANOVA without the missing data.
I have done a 2-way repeated measures ANOVA on the first 3 columns of the table excluding the gesturing condition, but this is unsatisfactory because I want to compare the Robot-Gesturing to the other Wand feedback methods.
I have spoken to people in my office who suggested testing for a difference between the Robot and Wand condition first, then testing for differences in feedback methods for the Robot condition and then for the wand condition independently. This has the same problem where I can't compare the Gesturing feedback to any of the Wand conditions.
What I would like to do is say that I am not considering the device (Robot/wand) as an independent variable, but instead consider each of the 7 modes as independent. In reality there is a correlation in performance between the Robot/wand so I don't think this is valid. However, is there a correction I perform that takes this correlation into account so that I can use 1-way repeated measures ANOVA and still perform pairwise comparisons between all the different conditions?
Thanks for you attention.
Appendix:
The paper is over the attachment limit so this is a dropbox link: https://www.dropbox.com/s/0xmebzara9dacrq/FeedbackPostReview.pdf?dl=0
An extract from the reviewer's comments:
"This study has 2 independent variables: 1) the device and 2) the feedback mode (this is called operation mode in Section III, which I believe is poorly named). Hence, the experimental design is 2-way. However, the analyses conducted in the paper utilize a 1-way ANOVA. This is technically incorrect. The authors need to report 2-way ANOVA results, which would clearly and reparately indicate whether there is a significant effect of feedback method and/or the device choice."