Compare 2 tests - Bland Altman vs. area under curve

[finnakerstrom]

Hello,

I want to compare the performance of 2 binary classification tests (positive vs. negative diagnosis) from the same sample (n=430). I have read about the Bland-Altman method and also about comparing the area under the ROC curves. Although the two methods of comparison are completely different I am not sure which one to use.

Many thanks

 

[hlsmith]

Can you tell us about the tests and data in more detail, along with your goals. I am assuming you have another variable that is the gold standard?

 

[finnakerstrom]

Sorry for insufficient information. The test is used to differentiate tachycardia A (positive) from tachycardia B (negative) using a time interval (ms) obtained from a novel pacing maneuver during electrophysiologic study. I have found an optimar cutoff of >80ms to differentiate tachycardia A from tachycardia B (ROC =0.97). Now there are many other pacing maneuvers that tries to distinguish tachycardia A from tachycardia B on the basis of time intervals (ms) and I want to compare their performance i.e. see if they work as well, better or worse than my novel pacing maneuver. However, there is no accepted gold standard method. In my study I have tested 2 other pacing maneuvers. Really, it is not so important for me to show that my method is better since its strength lies in its practical simplicity compared to the other methods, but I guess I need to do some kind of comparison.

 

[hlsmith]

How did you find an optimal cutoff if you don't have a gold standard?

 

[finnakerstrom]

I took the cutoff that resulted in the highest sum of sensitivity+specificity... maybe that's incorrect? I should also add that to reach a definite diagnosis of tachycardia A (positive) or tachycardia B (negative) one usually needs to perform several pacing maneuvers and integrate all that information along with the response to therapy.

 

[hlsmith]

So the test was related to the evaluation of the test? To get SEN of SPEC you need a gold standard or a iffy proxy.

 

[finnakerstrom]

The test is not related to the evaluation of the test as far as I am concerned. The definite diagnosis (gold standard) of tachycardia A or tachycardia B is a compound of several observations: clinical observations, ECGs, different measuremente during electrophysiologic study (not the measuremente I'm looking at), response to treatment etc. There is currently no single measuremente during electrophysiologic study that has 100% sensitivity/specificity.