L1 regularization, or LASSO (Least Absolute Shrinkage and Selection Operator), is a kind of regularization in which the penalty is in the form of the absolute magnitude of the coefficients. The cost function in the L1 setup is as follows, where lambda is the regularization parameter. Larger values of lambda correspond to more regularization and thus a simpler model with smaller coefficients. Note that if lambda is set to 0, the problem reduces to ordinary least squares, and no regularization is applied.
In LASSO regression, the coefficients can be shrunk all the way to 0 for predictors that have no relationship with the response. Thus, LASSO can be used as a form of variable selection, in which the least important predictors are eliminated from the regression model.
