After a long search I think I finally got this right, but would like to know for sure. I've been trying to sort out all the variations on the McNemar test. What I've found so far:
McNemar test as proposed by McNemar himself in 1947 uses a chi-square distribution. As such a continuity correction could sometimes be used (either the standard Yates (1934), or Edwards (1948)).
If the number of cells in the diagonal is low (usually below 25) the McNemar test becomes unreliable and an exact test would be preferred. This can either be done using the binomial test, or an F-test. The exact test using the F-distribution is known as Liddell's exact test (1983).
Finally Fagerland, Lydersen & Laake (2013) showed that the best approach might be a mid-p value.
So in conclusion: forget about all the McNemar test and all it's variation, and simply use the mid-p value.
Is my final conclusion correct?
Below are the formula's for each of them.
Given a 2x2 cross table with cells:
a b
c d
McNemar:
Chi-square value of (b - c)^2 / (b + c) and df = 1.
McNemar with Yates correction:
Chi-square value of (|b - c| - 0.5)^2 / (b + c) and df = 1.
McNemar with Edwards correction:
Chi-square value of (|b - c| - 1)^2 / (b + c) and df = 1.
Binomial exact test:
number of trials is min(b, c), trials = b + c, prob. = 0.5
Liddell's exact test:
F = b/(c + 1), df1 = 2(c + 1), df2 = 2b.
mid-p value:
Exact p-value - combin(b+c, c)*0.5^b*0.5^(n-b)
Most of the above formula's except the Yates corrected and Liddell's exact test can be found on the wikipedia entry: http://en.wikipedia.org/wiki/McNemar's_test
McNemar test as proposed by McNemar himself in 1947 uses a chi-square distribution. As such a continuity correction could sometimes be used (either the standard Yates (1934), or Edwards (1948)).
If the number of cells in the diagonal is low (usually below 25) the McNemar test becomes unreliable and an exact test would be preferred. This can either be done using the binomial test, or an F-test. The exact test using the F-distribution is known as Liddell's exact test (1983).
Finally Fagerland, Lydersen & Laake (2013) showed that the best approach might be a mid-p value.
So in conclusion: forget about all the McNemar test and all it's variation, and simply use the mid-p value.
Is my final conclusion correct?
Below are the formula's for each of them.
Given a 2x2 cross table with cells:
a b
c d
McNemar:
Chi-square value of (b - c)^2 / (b + c) and df = 1.
McNemar with Yates correction:
Chi-square value of (|b - c| - 0.5)^2 / (b + c) and df = 1.
McNemar with Edwards correction:
Chi-square value of (|b - c| - 1)^2 / (b + c) and df = 1.
Binomial exact test:
number of trials is min(b, c), trials = b + c, prob. = 0.5
Liddell's exact test:
F = b/(c + 1), df1 = 2(c + 1), df2 = 2b.
mid-p value:
Exact p-value - combin(b+c, c)*0.5^b*0.5^(n-b)
Most of the above formula's except the Yates corrected and Liddell's exact test can be found on the wikipedia entry: http://en.wikipedia.org/wiki/McNemar's_test
