Dear Forum,
I encountered an employee satisfaction survey report which sampled 50,000 employees (try not to laugh, that's a sample not the population). The authors reported % agreement based on Likert scales, which is not great, but ok, and compared said "scores" to a normative benchmark (for example 65% of respondents agree with this statement, 35% disagree; norm is 83% agree). However, the part that threw me off was that they proclaimed that a 7% difference from the benchmark for any score is considered meaningful. Certainly, this is not the margin of error, not with n = 50,000. I've seen this before in other surveys where someone will state in a survey that any score above x% is a meaningful difference - I can only assume that they mean x to be the margin of error.
Can anyone explain how a 7% difference in score could possibly be "significant" / meaningful on n = 50,000? The only thing I could think of was the average total score change year over year... but that seems far-fetched.
so, either I missed something in survey stats 101, or that whole 7% thing is completely made up.
Thanks!
I encountered an employee satisfaction survey report which sampled 50,000 employees (try not to laugh, that's a sample not the population). The authors reported % agreement based on Likert scales, which is not great, but ok, and compared said "scores" to a normative benchmark (for example 65% of respondents agree with this statement, 35% disagree; norm is 83% agree). However, the part that threw me off was that they proclaimed that a 7% difference from the benchmark for any score is considered meaningful. Certainly, this is not the margin of error, not with n = 50,000. I've seen this before in other surveys where someone will state in a survey that any score above x% is a meaningful difference - I can only assume that they mean x to be the margin of error.
Can anyone explain how a 7% difference in score could possibly be "significant" / meaningful on n = 50,000? The only thing I could think of was the average total score change year over year... but that seems far-fetched.
so, either I missed something in survey stats 101, or that whole 7% thing is completely made up.
Thanks!
Last edited: