Hello all,
Right off the bat I'd like to say that I'm pretty new to statistics and am having to employ some for analysing the results of a listening test I conducted for my masters thesis (in Environmental Acoustics), so apologies if I struggle to explain myself clearly!
Essentially, my listening test involved subjects listening to seven different stimuli and adjusting their levels so that they give a perception of equal annoyance relative to a reference stimuli.
So, my results have the adjustment levels (in dB) of 15 participants, for each of the 7 stimuli.
My objective is to find out how combined noise characteristics affect annoyance - i.e if someone adjusts a stimuli with a tone by -3dB and another with an impulsive characteristic by -6bB (against the reference) then how much do they adjust the stimuli that includes both the impulsive and tonal characteristic combined?
This has really glossed over my experiment and some fundamental acoustic concepts that you probably wouldn't need to know especially, but if any more info is needed or you're simply interested please let me know...
I'm using a combination of Matlab and Excel for analysing my results.
After plotting the means and errorbars for 95% confidence (some of which are quite large - I've attached the plot) I performed a one-way ANOVA on all of the results, which showed there was significance somewhere within them with a p-value of 0.010234!
I followed that up with a multcompare analysis in Matlab, (which essentially is performing Tukey's HSD test), which showed no significance between groups (stimuli)! This conflicts what the ANOVA was saying, sort of. Though I have a limited understanding that they're kind of looking at different things - so it is possible!
My supervisor and I were a bit confused by this, so did a bit of reading and some of the assumptions were not met for the ANOVA and Tukey's test - turns out that 16 of the 42 condition comparisons do not have equal variance. Also tested for equal distribution for each stimulus using the Lilliefors test and 4 of the 7 conditions were shown to not be normally distributed. A QQ plot in Matlab (attached) indicated the same thing for all results across all stimuli.
So, I was then going to look at doing a t-test for unequal variance but realised I couldn't due to the normal distribution assumption not being met. So then I figured perhaps a non-parametric t-test equivalent would be the way to go; perhaps the Mann-Whitney test, however I then stumbled across this when learning about non-parametric tests:
"Don't be too quick to switch to using the nonparametric Kruskal-Wallis ANOVA (or the Mann-Whitney test when comparing two groups). While nonparametric tests do not assume Gaussian distributions, the Kruskal-Wallis and Mann-Whitney tests do assume that the shape of the data distribution is the same in each group. So if your groups have very different standard deviations and so are not appropriate for one-way ANOVA, they also should not be analyzed by the Kruskal-Wallis or Mann-Whitney test."
I took a look at the standard deviation from the scores of each stimuli and they do vary quite a bit - 3.2, 2.7, 2.5, 4.3, 5.3, 4.7 and 5.2. So I assume that I can't use nonparametric tests after all?
I was looking at transforming my data, though after chatting with my supervisor today he suggested that I look at performing t-tests with a p-value correction for multiple comparisons? But surely that wouldn't be appropriate as the assumptions for the t-test still are not met?
If I did need to transform the data it probably wouldn't be wise to do a log transformation as the adjustments are in a dB (hence logarithmic) scale anyway!
Any pointers as to where to go from here (if you've survived reading for this long) would be greatly appreciated! Also, please let me know if any further information is needed or if I've not explained myself very well!
I hope this is the correct forum to post in as well, though there is a possibility this might need to be moved to the applied statistics forums - hopefully the mods can move it there if that's the case.
Thank you in advance for any help or suggestions.
Right off the bat I'd like to say that I'm pretty new to statistics and am having to employ some for analysing the results of a listening test I conducted for my masters thesis (in Environmental Acoustics), so apologies if I struggle to explain myself clearly!
Essentially, my listening test involved subjects listening to seven different stimuli and adjusting their levels so that they give a perception of equal annoyance relative to a reference stimuli.
So, my results have the adjustment levels (in dB) of 15 participants, for each of the 7 stimuli.
My objective is to find out how combined noise characteristics affect annoyance - i.e if someone adjusts a stimuli with a tone by -3dB and another with an impulsive characteristic by -6bB (against the reference) then how much do they adjust the stimuli that includes both the impulsive and tonal characteristic combined?
This has really glossed over my experiment and some fundamental acoustic concepts that you probably wouldn't need to know especially, but if any more info is needed or you're simply interested please let me know...
I'm using a combination of Matlab and Excel for analysing my results.
After plotting the means and errorbars for 95% confidence (some of which are quite large - I've attached the plot) I performed a one-way ANOVA on all of the results, which showed there was significance somewhere within them with a p-value of 0.010234!
I followed that up with a multcompare analysis in Matlab, (which essentially is performing Tukey's HSD test), which showed no significance between groups (stimuli)! This conflicts what the ANOVA was saying, sort of. Though I have a limited understanding that they're kind of looking at different things - so it is possible!
My supervisor and I were a bit confused by this, so did a bit of reading and some of the assumptions were not met for the ANOVA and Tukey's test - turns out that 16 of the 42 condition comparisons do not have equal variance. Also tested for equal distribution for each stimulus using the Lilliefors test and 4 of the 7 conditions were shown to not be normally distributed. A QQ plot in Matlab (attached) indicated the same thing for all results across all stimuli.
So, I was then going to look at doing a t-test for unequal variance but realised I couldn't due to the normal distribution assumption not being met. So then I figured perhaps a non-parametric t-test equivalent would be the way to go; perhaps the Mann-Whitney test, however I then stumbled across this when learning about non-parametric tests:
"Don't be too quick to switch to using the nonparametric Kruskal-Wallis ANOVA (or the Mann-Whitney test when comparing two groups). While nonparametric tests do not assume Gaussian distributions, the Kruskal-Wallis and Mann-Whitney tests do assume that the shape of the data distribution is the same in each group. So if your groups have very different standard deviations and so are not appropriate for one-way ANOVA, they also should not be analyzed by the Kruskal-Wallis or Mann-Whitney test."
I took a look at the standard deviation from the scores of each stimuli and they do vary quite a bit - 3.2, 2.7, 2.5, 4.3, 5.3, 4.7 and 5.2. So I assume that I can't use nonparametric tests after all?
I was looking at transforming my data, though after chatting with my supervisor today he suggested that I look at performing t-tests with a p-value correction for multiple comparisons? But surely that wouldn't be appropriate as the assumptions for the t-test still are not met?
If I did need to transform the data it probably wouldn't be wise to do a log transformation as the adjustments are in a dB (hence logarithmic) scale anyway!
Any pointers as to where to go from here (if you've survived reading for this long) would be greatly appreciated! Also, please let me know if any further information is needed or if I've not explained myself very well!
I hope this is the correct forum to post in as well, though there is a possibility this might need to be moved to the applied statistics forums - hopefully the mods can move it there if that's the case.
Thank you in advance for any help or suggestions.
