Logistic regression using Lasso
- Thread starter noetsi
- Start date
Why dont you have a linear model? Also, if you do what @GretaGarbo mentioned you need to perform group lasso to ensure it is grabbing dependent variables together.
But of course it is a linear model. It is a generalized linear model. It is a logistic multiple linear regression, like this:
log(p/(1-p)) = beta_0 + beta_1*x_1 + beta_2*x_2
It is linear in the parameter. i.e. the beta:s.
But it has a non-linear link function: the stuff with "log(p/(1-p))". If you solve that equation, that is maninulate it so that you only get "p" on the left hand side, then you will get:
p = (exp(beta_0 + beta_1*x_1 + beta_2*x_2))/(1 + exp(beta_0 + beta_1*x_1 + beta_2*x_2))
If you plot that curve against x_1 (let x_2 be a constant) you will get an S-shaped curve.
You can check if the factor levels of x_1 are approximately on a line:
log(p/(1-p)) = beta_0 + factor(x_1)_i + beta_2*x_2
Anyway, it will maybe be a good enough approximation.
log(p/(1-p)) = beta_0 + beta_1*x_1 + beta_2*x_2
It is linear in the parameter. i.e. the beta:s.
But it has a non-linear link function: the stuff with "log(p/(1-p))". If you solve that equation, that is maninulate it so that you only get "p" on the left hand side, then you will get:
p = (exp(beta_0 + beta_1*x_1 + beta_2*x_2))/(1 + exp(beta_0 + beta_1*x_1 + beta_2*x_2))
If you plot that curve against x_1 (let x_2 be a constant) you will get an S-shaped curve.
You can check if the factor levels of x_1 are approximately on a line:
log(p/(1-p)) = beta_0 + factor(x_1)_i + beta_2*x_2
Anyway, it will maybe be a good enough approximation.
Thanks GretaGarbo, I always think of linear models as ones that have interval DV 
I never think of logistic regression as linear, but I understand your point.
But my original point remains. SAS's documentation says that its LASSO function was not designed for Logistic regression results. I will try to find some examples of their comments.
I never think of logistic regression as linear, but I understand your point.But my original point remains. SAS's documentation says that its LASSO function was not designed for Logistic regression results. I will try to find some examples of their comments.
I just google searched a few of these words and found this.
But don't make it complicated. If Lasso is to complicated at the moment so drop it. Do some linear regressions and choose a model.
But don't make it complicated. If Lasso is to complicated at the moment so drop it. Do some linear regressions and choose a model.