What is ElasticNet Regression 

Def: Elastic Net Regression is a powerful machine learning algorithm that combines the features of both Lasso and Ridge Regression. It is a regularized regression technique that is used to deal with the problems of multicollinearity and overfitting, which are common in high-dimensional datasets. This algorithm works by adding a penalty term to the standard least-squares objective function [1].In simple terms, the elastic net regression took the qualities of ridge and lasso regression to regularize the machine learning regression mode [2]l.

Elastic Net Regression was introduced by Zou and Hastie in 2005. It is a linear regression algorithm that adds two penalty terms to the standard least-squares objective function. These two penalty terms are the L1 and L2 norms of the coefficient vector, which are multiplied by two hyperparameters, alpha and lambda. The L1 norm is used to perform feature selection, whereas the L2 norm is used to perform feature shrinkage [1].

The Elastic Net Regression model can be represented as follows :

y = b0 + b1*x1 + b2*x2 + ... + bn*xn + e

Where y is the dependent variable, b0 is the intercept, b1 to bn are the regression coefficients, x1 to xn are the independent variables, and e is the error term. The Elastic Net Regression model tries to minimize the following objective function:

RSS + λ * [(1 - α) * ||β||2 + α * ||β||1]

Where RSS is the residual sum of squares, λ is the regularization parameter, β is the coefficient vector, α is the mixing parameter between the L1 and L2 norms, ||β||2 is the L2 norm of β, and ||β||1 is the L1 norm of β.

In ridge regression, we add extra terms in the loss function that will help the slope value for minimized over-fit[2]

The loss function becomes:

L = sum(Yi — Yi_hat)² + lambda(w²)

Where,

lambda = penalty term (a constant value can be chosen) W = coefficients of the features

Advantages 

Where do we use elastic net regression [2]

• It helps to overcome the issues of over-fitting with ridge quality.
• Dealing with multi-collinearity issues in the data.
• Reducing features in the data with lasso quality.

Disadvantages 

Elastic Net Regression in Python 

Elastic net regression is implemented in the scikit-learn library for Python using the ElasticNet class. The ElasticNet class has two main hyperparameters:

• alpha: Controls the overall strength of the regularization penalty. A larger value of alpha will result in more shrinkage of the coefficients.
• l1_ratio: Controls the balance between the L1 and L2 regularization penalties. A value of l1_ratio=1 corresponds to pure lasso regularization, while a value of l1_ratio=0 corresponds to pure ridge regularization.

The following code shows how to use the ElasticNet class to fit a model to a dataset of house prices:

from sklearn.linear_model import ElasticNet 
# Load the data data = pd.read_csv("house_prices.csv") 
# Split the data into features and target 
features = data.drop("price", axis=1)
target = data["price"]
 # Fit the ElasticNet model 
model = ElasticNet(alpha=1.0, l1_ratio=0.5).fit(features, target) 
# Print the model coefficients 
print(model.coef_)

from sklearn.linear_model import LassoCV, RidgeCV, ElasticNet
from sklearn.model_selection import cross_val_score
#Implementation of ElasticNet
elastic = ElasticNet(alpha=0.001)
print("Root Mean Squared Error (ElasticNet): ", np.sqrt(-cross_val_score(elastic, X, y, cv=10, scoring='neg_mean_squared_error')).mean())

Reference

1-Elastic Net Regression detailed guide!

2- Fully Understand ElasticNet Regression with Python