Interpreting sensitivity and specificity with corresponding PPV and NPV

[amb_e]

Hello,

I have calculated sensitivities, specificities with pos and neg predictive values for different cutoff values of two biomarkers.

What i get is the following:

Biomarker 1: High spec (92%) low sens (20%) with high NPV (91%) and low PPV (22%)
Biomarker 2: High sens (90%) low spec (30%) with high NPV (96%) and low PPV (13%)

Does the high NPV for both biomarkers mean that they can be used to rule out the disease, or do you need to take in consideration the corresponding sensitivity and specificity ?
Whats is the best way to interpret these numbers ?

Thanks !

 

[hlsmith]

Did you report your values correctly above? Typically the SEN and NPV are influenced by the false negative value and the reciprocal is true for SPEC and PPV and the false positive value. Data seem not to convey this.

What was the true prevalence of the condition under investigation.

 

[amb_e]

I checked the data. I still get the same results.

Can you explain the part of how SEN and NPV are influenced by the negative value and visa versa ?
Do you mean a high SEN should correspond with a high NPV and a high SPEC with a high PPV ?
For biomarker 1: false positive was 0.77 and false negative 0.09
Biomarker 2: false positive was 0.87 and false negative 0.04.

The prevalence was 10%.
Analyses was done in a small group, but that wouldn't have any effect on SEN, SPEC etc right ?

 

[hlsmith]

Look at your post one more time, and confirm your numbers are correct:

Biomarker 1: High spec (92%) low sens (20%) with high NPV (91%) and low PPV (22%)
Biomarker 2: High sens (90%) low spec (30%) with high NPV (96%) and low PPV (13%)

Intuitively these red groups seem switched.

Yes, the false negatives are in the denominator for both the Sen and NPV. So the higher the value, then the lower those values (SEN and NPV) become. Same thing holds for FP and the SPEC and PPV.

In general, they say these metrics are best when your prevalence is between 20-80%. One reason is when you come across extremely low or high prevalences, if there are very low or high disconcordant values (false positives or negatives) then those values either carry too little or much weight in the forementioned denominators.

 

[hlsmith]

If the small groups were good representations of the population it should not matter too much. Were they randomly selected? Though, small groups can impact the breadth of your confidence intervals. Fewer people = wider confidence intervals and vice versa.

 

[amb_e]

Not randomly selected. The group consists of patients with a disease and we are looking at the biomarkers, whether they can predict a certain outcome. And 10% had the outcome.
These SENS, SPEC, PPV and NPV are calculated based on cutoff values published in literature. But for the first biomarker, 80% of the patients with the outcome we where interested in had levels below this cutoff value. So that was unexpected.

So what I did afterwards is calculate new optimal cutoff values for this group (I used ROC curves)
The new cutoff value gave me SENS of 60%, SPEC 81 % PPV 26% NPV 95 % (biomarker 1)
and SENS 70% SPEC 60% PPV 17% NPV 91% (biomarker 2)

 

[amb_e]

So now I have metrics of both biomarkers, first based on literature and secondly based on optimal cutoff for this group.

 

[hlsmith]

Interesting, plus you have your area under the curve (AUC) values. I would recommend adding confidence intervals around all of these measures. Lastly, some programs (e.g., SAS) will allow you to compare the two AUC values for the biomarkers to tell whether they may be statistically different based on their concordance (AUC).

You would also want to examine your patient sample characteristics to those used to construct the cut-off from the literature in order to see if there are any differences. One thing in particular may be the severity of the illness or other variables that may influence outcomes of interest (e.g., age, comorbidities, etc.).

 

[amb_e]

Thanks !

Im using SPSS, which doesn't support the AUC comparing. Based on the 95 CI only the newly calculated cutoff for biomarker 1 could significantly predict the outcome. ( 0.5 not being in the 95 CI for AUC and p value < 0.05).

 

[hlsmith]

You can probably slap some 95% CIs on both AUCs and get a general idea if one may be better, if its 95% CI don't include the other's AUC value.