Learning and Validation Curves in Scikit-Learn: A Beginner-Friendly Guide

Ever wondered how to know if your machine learning model is underfitting or overfitting?

That’s where learning curves and validation curves come in. They give you visual feedback on how well your model is learning — and whether it might need more data, less complexity, or better tuning.

In this post, we’ll break down what these curves are and how to generate them using Python and scikit-learn. We’ll use the built-in digits dataset, so you can run the code straight away.

🧪 Learning Curve

A learning curve shows how the model performance changes with the size of the training set. It helps answer questions like:

• Should I get more training data?

• Is my model underfitting?

• Has my model already learned all it can?

🧪 Python Code: Learning Curve Example

import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import learning_curve
from sklearn.model_selection import ShuffleSplit
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
# Load dataset
X, y = load_digits(return_X_y=True)
# Create a pipeline with standard scaling and logistic regression
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
# Set up cross-validation
cv = ShuffleSplit(n_splits=5, test_size=0.2, random_state=42)
# Generate learning curve
train_sizes, train_scores, test_scores = learning_curve(
 model, X, y, cv=cv, scoring='accuracy',
 train_sizes=[0.1, 0.3, 0.5, 0.7, 1.0], random_state=42
)
# Calculate average and standard deviation
train_mean = train_scores.mean(axis=1)
train_std = train_scores.std(axis=1)
test_mean = test_scores.mean(axis=1)
test_std = test_scores.std(axis=1)
# Plot learning curve
plt.figure(figsize=(8, 5))
plt.plot(train_sizes, train_mean, label="Training Score")
plt.fill_between(train_sizes, train_mean - train_std, train_mean + train_std, alpha=0.2)
plt.plot(train_sizes, test_mean, label="Cross-Validation Score")
plt.fill_between(train_sizes, test_mean - test_std, test_mean + test_std, alpha=0.2)
plt.xlabel("Training Set Size")
plt.ylabel("Accuracy")
plt.title("Learning Curve - Logistic Regression on Digits Dataset")

🔧 Validation Curve

Using the cross-validation method, the validation curve function evaluates training and test performance over the range of hyperparameter values. The process results in two sets of scores that we can compare visually.

Using the cross-validation method, the validation curve function evaluates training and test performance over the range of hyperparameter values. The process results in two sets of scores that we can compare visually.

The function explores machine learning model performance over various values of specific hyperparameters.

A validation curve helps you understand how changing a hyperparameter (like the depth of a tree or number of neighbors) affects performance. It tells you:

• Is my model too simple or too complex?

• What’s the best value for this parameter?

import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import validation_curve
from sklearn.svm import SVC
from sklearn.datasets import load_iris
data = load_iris()
X, y = data.data, data.target
param_range = np.logspace(-3, 3, 5) 
train_scores, test_scores = validation_curve(
 SVC(), X, y, 
 param_name="gamma", 
 param_range=param_range,
 cv=5, 
 scoring="accuracy"
)

✅ Summary: When to Use These Curves

References