Regression Using PyCaret: A Simplified Approach

Introduction 

Regression analysis is a set of statistical processes for estimating the relationships between a dependent variable (often called the 'outcome variable', or 'target') and one or more independent variables (often called 'features', 'predictors', or 'covariates'). The objective of regression in machine learning is to predict continuous values such as sales amount, quantity, temperature etc.

Regression analysis is a fundamental technique in machine learning used to predict numerical values based on input features. Traditionally, setting up a regression model requires extensive preprocessing, model selection, and hyperparameter tuning. However, PyCaret, an open-source low-code machine learning library, simplifies this process by automating tasks such as feature engineering, model training, and evaluation.

In this blog, we will walk through how to perform regression using PyCaret with minimal code and effort.

Why Use PyCaret for Regression?

• Low-code Implementation: Reduces the amount of code needed for model building.

• Automated Preprocessing: Handles missing values, feature encoding, and transformations.

• Multiple Model Training: Trains and compares multiple regression models automatically.

• Hyperparameter Tuning: Provides an easy way to tune model parameters.

• Deployment Ready: Allows exporting trained models for production use.

1-Import library

import numpy as np 
import pandas as pd 
import seaborn as sns 
import matplotlib.pyplot as plt
from prettytable import PrettyTable
from sklearn.metrics import roc_curve, auc
from mlxtend.plotting import plot_confusion_matrix 
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, confusion_matrix
import warnings
warnings.filterwarnings("ignore")
#from pycaret.utils import enable_colab
#enable_colab()

2- Installing Pycaret

Before we begin, install PyCaret using pip:

#capture #suppresses the displays
# install the full version
!pip install pycaret[full]

!pip install pyyaml==5.4.1

3- Import the necessary packages

!pip install markupsafe==2.0.1

Runtime> Restart Runtime

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pycaret
import jinja2
from pycaret.regression import*

4-Dataset

For this tutorial, we will use the Boston Housing Dataset, which contains various housing-related features and a target variable (MEDV, median house price).

import pandas as pd
from pycaret.regression import *

# Load dataset
from sklearn.datasets import fetch_california_housing
data = fetch_california_housing(as_frame=True)
df = data.frame

# Display first few rows
df.head()

5- Setting up Environment in PyCaret

PyCaret requires setting up the environment using the setup() function. This function automatically handles preprocessing tasks.

# Initialize PyCaret regression setup
regression_setup = setup(data=df, target='MedHouseVal', session_id=123)

• data: The dataset to be used.

• target: The column we want to predict.

• session_id: Used for reproducibility.

6- Comparing All Models

This function trains all the available models in the model library using default hyperparameters and evaluates performance metrics using cross-validation. The number of folds can be defined using the foldparameter (default = 10 folds). The table is sorted (highest to lowest) by the metric of choice which can be defined using the sortparameter(in this case we have sorted it on RMSE) n_select parameter in the setup function controls the return of trained models. In this case, I am setting it to 15, meaning return the top 15 models as a list. pull function in the second line stores the output of compare_models as pd.DataFrame .

One of PyCaret’s powerful features is the compare_models() function, which evaluates multiple models and ranks them based on performance.

compare_models()

 
compare_models(sort = 'RMSE')

compare_models(fold = 5)

This command automatically trains and evaluates different regression models and selects the best-performing one.

7- Select Best Model

best= compare_models(n_select = 2, sort= 'RMSE')

compare_model_result = pull()

catboost = create_model('catboost')

print(catboost)

tuned_dt = tune_model('catboost')

tuned_lda= tune_model(catboost, optimize='RMSE', search_library='optuna')

#tuned model object is stored in the variable 'tuned_dt'. 
print(tuned_lda)

10-Plot a Model

plot_model(catboost, plot = 'residuals_interactive')

plot_model(catboost)

plot_model(catboost, plot='feature')

plot_model(catboost, plot = 'error')

predict_model(catboost);

unseen_predictions = predict_model(catboost, data=data_unseen)
unseen_predictions.head()

final_catboost = finalize_model(catboost)

#Final Light Gradient Boosting Machine parameters for deployment
print(final_catboost)

save_model(final_catboost,'Final Lightgbm Model 08Feb2020')

saved_final_catboost = load_model('Final catboost Model 08Feb2020')

Conclusion

PyCaret significantly simplifies the regression modeling process by automating various steps such as preprocessing, model comparison, tuning, and evaluation. This makes it an excellent choice for beginners and professionals who want to quickly build and deploy machine learning models.

Would you like to explore PyCaret further? Try it out with different datasets and advanced configurations!

References

1-Regression Tutorial (REG101) - Level Beginner

2- Regression_With_Pycaret.ipynb