Systematic way to find cases with random answers in a survey data set

[pvincentruz]

Hi everyone !

I'm new to the forums and I am a first year PhD student.

We just collected data with a survey the research team I work on created. We started making a few tests to see if everything was working all right. We have data about 1200 middle school kids in 6th and 8th grade

The whole survey consists on likert scale items. While doing IRT we noticed a few of our items were behaving strange. (People with high skill tend to choose an option that we expected would be answered by low skill kids)

We have about 8 items all are coded 1-4 (NO!,no,yes,YES!) However, half of the items could be reverse coded. so we expected to it to generate two different factor loadings in factor analysis. That did happen. They also correlate negatively to the others. But if you force the factor loading to 1 they fit. We think there are some cases that have answered randomly and that is why the IRT shows those items in a weird way. I managed to find about 10 cases that only answered "a" or "b" the whole survey but I was wondering if with IRT or another method one could find other cases where kids answered randomly and it is affecting out results.

Thanks for reading this. I am ELL so sorry if it is confusing.

 

[hlsmith]

The funny thing is, I am guessing, there is no correct answer beyond the truth that you can check for and it is too late to scramble response options up. I have never seen a (NO!,no,yes,YES!) scale, hmm. I don't have an answer for you on how to sleuth out your oddities, however there is much literature out there on survey instrument methodology - so there should be something. It reminds me of pursuits to detect missing completely at random or biases of missing data and checking for patterns.

Good luck.

 

[pvincentruz]

It reminds me of pursuits to detect missing completely at random or biases of missing data and checking for patterns.

Good luck.

Do you have any book you would recommend for that ?

P.S The scale is coded as YES!=4 and NO!=1
Thanks !

 

[hlsmith]

Unfortunately I am not that seasoned in MCAR examination. A typical approach is comparing their other values or characteristics to see if they vary from others in some regard.