About binary logistic regression models -confusing results of confusion matrices.

Member
Dear all,
I fit a binary logistic regression model obtain information on sens, spec, ppv, npv, CA. After this, i perform the 10-fold cross validation of the same model, and somehow my sens, spec, ppv, npv, CA are exactly the same.

Code:
#My codes look like this:
set.seed(2021)
#Full model
glm_model <- glm(outcome ~ ., df1, family = binomial(link = "logit"))
summary(glm_model)
res_glm_model <- predict(glm_model, df1, type = "response" )
(table(predictedvalue = res_glm_model > 0.5, Actualvalue = df1$outcome )) #Cross validation of the model. glm_crossValSettings <- trainControl(method = "cv", number = 10, savePredictions = "final") crossVal_glm<- train(glm_model, data = df1, family = "binomial", method = "glm", trControl = glm_crossValSettings, tuneLength = 2) pred_glm <- predict(crossVal_glm, newdata = df1) confusionMatrix(data = pred_glm, df1$outcome)
My problem is that the results on (table(predictedvalue = res_glm_model > 0.5, Actualvalue = df1$outcome )) and cross-validated confusionMatrix(data = pred_glm, df1$outcome) are exactly the same. Namely:

I can not understand where i am making the mistake, or can it be like this?

Thank you for the answers.

Last edited by a moderator:

hlsmith

Less is more. Stay pure. Stay poor.
I haven't ran this code yet, but what are you hoping for this code/approach to do or reveal? Obviously, taking estimates for generalization from a model that selected them is at risk for type I errors, but what do you think the CV is doing and outputting in this code?