Quick Cronbach's Alpha Question
- Thread starter Aegislash
- Start date
- Tags cronbach's alpha outliers reliability
There is one variable in my study with a z-score of approximately 4.60 and the participant opted to enter the highest value for each of the scale. There was a clear pattern to their responses, so it seemed to be the best option to remove them. Of course, when I remove them it decreases the Cronbach's alpha but if I include them then is it inaccurate?
Its only inaccurate if they did not really feel the way they responded. Which is unknowable. It is common to select middle values in responding when you have no strong feelings on the topic on some issues according to the psychometric literature. In satisfaction surveys always responding your are satisfied or extremely satisfied when you really don't care is likely IMHO based on my own analysis. But I have never seen a finding that is true from an expert.
It's very difficult to say for sure, but their data aroused suspicion. Oddly, the participant's data supported my hypothesis so if it was an attempt to sabotage my dataset then they went about it the wrong way! In this scale, higher scores indicated a greater impairment of quality of life and they scored 78/80 in total which was considerably higher than other participants in the study.
Uhmmm... Interesting. And did you have situations like reverse-coded items and whatnot that would make you think the participant actually read the questions and purposefully marked the most extreme response? Or would you say you have more of a condition where someone just marked the same response over and over again? Like the people you mark "extremely agree" in everything or something like that.
But if this seemed an attempt to actively throw your data off then removing this person from the analysis does not seem entirely unwarranted.
But if this seemed an attempt to actively throw your data off then removing this person from the analysis does not seem entirely unwarranted.
Personally I doubt someone tried to throw off the survey. What is likely is that they did not really care and/or know what they felt, but were reluctant to admit either. So they gave a socially desirable answer, I guess you could argue alternately the fact they answer at all is the socially desirable phenomenon.
Sometimes people clue in future answers based on past questions. This is why you try to break up your questions so not to clue the reader in to a certain set of responses. For example if a responder has answered 4 times in a row the highest level [because they really meant that in those cases] and you give them a similarly worded question they might not read real careful the specific wording and just automatically answer the same way.
Sometimes people clue in future answers based on past questions. This is why you try to break up your questions so not to clue the reader in to a certain set of responses. For example if a responder has answered 4 times in a row the highest level [because they really meant that in those cases] and you give them a similarly worded question they might not read real careful the specific wording and just automatically answer the same way.
There was no reverse coding, so it was more along the lines of the person clicking the highest response for each variable. The variables describe various negative situations that a person would have to experience so it doesn't add up all in all. Their responses to the demographic variables also raised suspicion.
That's a valid point and it could very well be true. It's likely that they didn't care as you say, and more than likely they opted to race through the survey. It's a shame that there were no reverse coded items in the scale, it would have made it much easier to determine if there was foul play or not.
Well, if it looks like this is a clear case of someone fudging the data (e.g. responses that are overall difficult to reconcile, perhaps crazy demographics data like when people say they were born in 1900, etc.) then I do feel you can remove that person from your datafile before proceeding with your analysis and then just mention that you did this somewhere and why you did it.
The one time that pretty much everyone agrees you can remove outliers is when it involves a clerical mistake. Whether that applies to what spunky mentioned I do not know. Again I find it hard to believe someone would take the time to answer a survey just to mess it up (although given how often psych surveys are given to undergraduates in psych classes maybe that is being optimistic).
Perhaps not intentionally, but there is a very vast area of research within psychology called acquiescence bias that goes directly at this issue (i.e. people who mess up your surveys/questionnaires because they do not respond honestly for a variety of reasons).
Within the Classical Test Theory framework (and Cronbach's alpha comes from it) acquiescence bias adds to random error so you can throw these datapoints out. The key issue here is to show evidence that the respondent is, indeed, exhibiting acquiescence bias.
Within the Classical Test Theory framework (and Cronbach's alpha comes from it) acquiescence bias adds to random error so you can throw these datapoints out. The key issue here is to show evidence that the respondent is, indeed, exhibiting acquiescence bias.