Hi everyone,
I'm having some difficulty deciding how best to analyze the data from a project. I had 4 radiologist read CT scans looking for specific signs (mostly giving True/False answers) on two occasions separated by a washout period. The last two variables included a 'Questionable' category.
In calculating the intrarater reliability, I assume Kappa would suffice. However, with the interrater, I'm not sure. Someone suggested that Krippendorff's alpha could be used, and might be better than the multiple Kappa. I'm also not sure whether I should treat it as 8 observers, or rather 4 observers in which case I might have to combine the two readings (perhaps 2x0's = 0, 1x0 and 1x1 = 1, 2x1's =2) and then do a weighted analysis?
Thanks so much!
(I'm using SPSS 17)
I'm having some difficulty deciding how best to analyze the data from a project. I had 4 radiologist read CT scans looking for specific signs (mostly giving True/False answers) on two occasions separated by a washout period. The last two variables included a 'Questionable' category.
In calculating the intrarater reliability, I assume Kappa would suffice. However, with the interrater, I'm not sure. Someone suggested that Krippendorff's alpha could be used, and might be better than the multiple Kappa. I'm also not sure whether I should treat it as 8 observers, or rather 4 observers in which case I might have to combine the two readings (perhaps 2x0's = 0, 1x0 and 1x1 = 1, 2x1's =2) and then do a weighted analysis?
Thanks so much!
(I'm using SPSS 17)