How to Configure Samplers¶

This guide shows you how to choose and configure Optuna samplers for your hyperparameter search. Use this when you want to control the optimization algorithm.

Prerequisites¶

Sklearn-Optuna installed (Getting Started)
A working OptunaSearchCV search

Interactive version available

Try this guide as an interactive notebook: View · Open in marimo

Choose a Sampler¶

Wrap any Optuna sampler with the Sampler class to make it compatible with Scikit-Learn's get_params() / set_params() / clone() API. See the Optuna samplers overview for a full comparison of available algorithms.

from sklearn_optuna import OptunaSearchCV, Sampler
import optuna

search = OptunaSearchCV(
    estimator, param_distributions,
    sampler=Sampler(sampler=optuna.samplers.TPESampler, seed=42),
)

TPE (default)¶

Tree-structured Parzen Estimator is the default when no sampler is specified. It works well for most search spaces:

sampler = Sampler(sampler=optuna.samplers.TPESampler, seed=42)

CMA-ES¶

CMA-ES works well for low-dimensional continuous search spaces. It is not suitable for categorical parameters:

sampler = Sampler(sampler=optuna.samplers.CmaEsSampler, seed=0)

Random¶

Use RandomSampler as a baseline comparison or when you want uniform coverage of the search space:

sampler = Sampler(sampler=optuna.samplers.RandomSampler, seed=0)

Set a Seed for Reproducibility¶

Pass seed= to the Sampler constructor. Results are deterministic when n_jobs=1:

search = OptunaSearchCV(
    estimator, param_distributions,
    sampler=Sampler(sampler=optuna.samplers.TPESampler, seed=42),
    n_jobs=1,
)

If you use n_jobs > 1, trial ordering is non-deterministic and results may vary between runs even with the same seed.

Use a Sampler in Nested Cross-Validation¶

Because the Sampler wrapper supports get_params() and set_params(), it survives clone() and can be used as a tunable hyperparameter:

from sklearn.model_selection import cross_val_score

search = OptunaSearchCV(
    estimator, param_distributions,
    sampler=Sampler(sampler=optuna.samplers.TPESampler, seed=42),
)

# The sampler is cloned correctly for each outer fold
scores = cross_val_score(search, X, y, cv=5)