Kappa for nonsquare tables

ondansetron

TS Contributor
#1
I have two raters who could have assigned any of 16 ratings to a subject. There are some categories that were not used, so the resulting table is not square and SAS will not give me the Kappa (Cohen's) for it.

I'm having trouble figuring out how to do this quickly and efficiently to get the Kappa statistic with some zero rows or columns. SAS notes some option about weighting with ZEROES but it seems I would need to basically reformat the data into the summarized counts of each possible outcome.

Anyone able to offer advice to tackle the problem? If I do need to summarize the counts for each category, what is the quickest way to do this for a 16x16 table?
 

hlsmith

Not a robit
#4
hmm, just too busy to look it up right now, this was my SAS code, not sure if it is applicable to you.

if this still doesn't work with uneven structure, I must have done it in R with another procedure:

Code:
proc freq data=epa2_rev;



table EPAn_S_1ID_sets * EPAn_SD_1ID_sets / agree (wt=fc) alpha=0.01;

exact agree;

run;
 

ondansetron

TS Contributor
#5
Yeah I didn't work, but STATA automatically handles it regardless of square or not, so I'm gonna squeak by with that for now. Thanks!

I had spent a while looking for it and couldn't come up with a way that worked for me.
 

spunky

Doesn't actually exist
#6
Yeah I didn't work, but STATA automatically handles it regardless of square or not, so I'm gonna squeak by with that for now. Thanks!.
Do you know that STATA does that it lets you handle it? Is it still fitting everything as if it were a square matrix and applying some sort of 'continuity correction'? I remember some software just throw in a 0.5 if there are 0s anywhere.
 

ondansetron

TS Contributor
#8
Do you know that STATA does that it lets you handle it? Is it still fitting everything as if it were a square matrix and applying some sort of 'continuity correction'? I remember some software just throw in a 0.5 if there are 0s anywhere.
Actually, I don't think I looked at how it's handled. That would've been smart. I'll try to read into it. Do you have any SAS recommendations?