What is the best way to include control variables in an Adaptive LASSO?

[superdayv]

Suppose I have several predictors of interest. I also have a variable that I think will be important to control for.

What is the best way to handle this in an Adaptive LASSO?

One answer I've come across is to set lamda to 0 for the control variable. I'm curious whether to do this for the initial model as well as the re-scaling model?

But I've also now read about double selection methods for LASSO (though I haven't seen this for an adaptive LASSO), as well as talk of just partialling out the control variable. Others have said they just throw the control variable(s) in with the predictors of interest and let them all get shrunk together.

I'm curious to hear if there is a generally accepted best way to deal with control variables in an Adaptive LASSO.

 

[hlsmith]

By adaptive LASSO approach, do you mean you are fitting ridge or GLM to get the starting coefficient values prior to using LASSO?

I haven't seen your question in the literature, trying to force a variable into the model. This partially comes up when there are interaction terms where the base terms need to be kept in the model, where this is addressed via group LASSO. My initial question, why are you using LASSO in the first place. I have a feeling in the long run LASSO may end-up being a glorified step regression, though its regularizing properties are great if one is looking for generalizability.

Why is the covariate important and what do you think it is associated with in the model that it needs to stay in?

My first inkling would be for the covariate to just go along for the full ride. But I would be happy to hear your thoughts or what else you may find in the literature.

 

[superdayv]

Hi thanks for the reply! I believe this approach is called the "General Form".

I am using LASSO as part of a battery of approaches. But mainly because of its tendency towards sparser models.

I am not certain that the variable matters, it is just one of those variables that tend to get controlled for.

My worry with letting it go for the full ride (I assumed you mean not forcing it in?) is I'm not sure the extent to which it would still be "controlled for" in that case?

 

[hlsmith]

By controlled for, you are speculating it may have a confounding effect on another variable and the outcome. Do you think it may be a common cause of multiple variables? It might help if you describe the context you are working in.

 

[hlsmith]

I was just thinking, there are also Bayesian LASSO approaches where you can provide priors to your variables. If you wanted to force a variable in, I would wonder if providing a sharp prior weight may force it into the model. However, the stipulation would be that you would want the weight to be based on real data, otherwise its estimate may be biased.