Introduction#

Optimization finds the best parameters from a set of possibilities. Evaluating every combination is usually impractical, so algorithms guide the search. Hyperactive provides a unified interface: define your problem once, then swap between different optimization algorithms without changing your code.

Why Hyperactive?#

Hyperactive makes optimization simple. Define your problem once, then swap between 31 different algorithms with a single line change.

Swap Algorithms

Try different optimizers without rewriting code. One line change switches from hill climbing to Bayesian optimization.

31 Algorithms

Local search, global search, population-based, and model-based methods. All with the same simple interface.

ML Integrations

Ready-to-use experiments for sklearn, sktime, skpro, and PyTorch. Tune models with minimal code.

Any Python Function

Works with simulations, engineering problems, or any function that returns a score.

Quick Start#

The simplest optimization in 5 lines:

from hyperactive.opt.gfo import HillClimbing


def score(p):
    return -(p["x"] ** 2)  # Find x that minimizes x²


opt = HillClimbing({"x": range(-10, 11)}, experiment=score)
best = opt.solve()  # {"x": 0}

ML Integration Examples#

Tune machine learning models with ready-to-use experiment classes:

from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.ensemble import GradientBoostingClassifier

experiment = SklearnCvExperiment(GradientBoostingClassifier(), X, y, cv=5)

from hyperactive.experiment.integrations import SktimeForecastingExperiment
from sktime.forecasting.arima import ARIMA

experiment = SktimeForecastingExperiment(ARIMA(), y_train, fh=[1, 2, 3])

from hyperactive.experiment.integrations import TorchTrainerExperiment
import pytorch_lightning as pl

experiment = TorchTrainerExperiment(YourLightningModule, train_loader)

See Framework Integrations for complete examples and all available integrations.

Core Concepts#

Hyperactive is built around three simple concepts:

Hyperactive core concepts: Experiment, Search Space, and Optimizer flow to Best Parameters Hyperactive core concepts: Experiment, Search Space, and Optimizer flow to Best Parameters

What to optimize

Experiment

Any Python function that takes parameters and returns a score. Hyperactive will maximize this score.

def experiment(params):
    return score

Learn more

Where to search

Search Space

A dictionary mapping parameter names to possible values. Defines the boundaries of your optimization.

{"x": [1, 2, 3], "y": ["a", "b"]}

Learn more

How to search

Optimizer

The algorithm that explores the search space. Choose based on your problem characteristics.

HillClimbing(space, experiment=exp)

Learn more

The Power of Swapping#

Define your problem once, then try different algorithms with a single line change:

from hyperactive.opt.gfo import HillClimbing

optimizer = HillClimbing(search_space, experiment=experiment)
best = optimizer.solve()

Fast local search. Good for quick exploration.


optimizer = BayesianOptimizer(search_space, experiment=experiment)
best = optimizer.solve()

Learns from past evaluations. Best for expensive functions.

from hyperactive.opt.gfo import GeneticAlgorithmOptimizer

optimizer = GeneticAlgorithmOptimizer(search_space, experiment=experiment)
best = optimizer.solve()

Population-based evolution. Great for complex landscapes.

Tip

The experiment and search space stay the same. Only the optimizer changes. This makes it easy to benchmark different algorithms on your problem.

Complete Example#

Here’s a full working example that tunes a Random Forest classifier:

import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier

from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.gfo import BayesianOptimizer

# 1. Load your data
X, y = load_iris(return_X_y=True)

# 2. Define the experiment (what to optimize)
experiment = SklearnCvExperiment(
    estimator=RandomForestClassifier(),
    X=X,
    y=y,
    cv=5,
)

# 3. Define the search space (where to search)
search_space = {
    "n_estimators": list(range(10, 200, 10)),
    "max_depth": [3, 5, 10, 20, None],
    "min_samples_split": [2, 5, 10],
}

# 4. Choose an optimizer (how to search)
optimizer = BayesianOptimizer(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
    random_state=42,
)

# 5. Run and get the best parameters
best_params = optimizer.solve()
print(f"Best parameters: {best_params}")

Common Parameters#

All optimizers share these parameters:

Parameter

Type

Description

search_space

dict

Maps parameter names to possible values

n_iter

int

Number of optimization iterations

experiment

callable

The objective function or experiment object

random_state

int

Seed for reproducibility

initialize

dict

Control initial population (warm starts, etc.)

Warm Starting#

You can provide starting points for optimization:

warm_start = [
    {"n_estimators": 100, "max_depth": 10},  # Start from known good point
]

optimizer = HillClimbing(
    search_space=search_space,
    n_iter=5,
    experiment=experiment,
    initialize={"warm_start": warm_start},
)

Tips for Beginners#

Start Simple

Begin with HillClimbing or RandomSearch to establish baselines before trying sophisticated algorithms.

Right-Size Your Space

Large search spaces need more iterations. Use np.logspace for parameters that span orders of magnitude.

Set Random State

For reproducible results, always set random_state=42 (or any integer).

Match Algorithm to Budget

Expensive evaluations? Use BayesianOptimizer which learns from each evaluation. Cheap evaluations? RandomSearch explores well.

Next Steps#

Experiments

Learn how to define what to optimize, including custom functions and built-in ML experiments.

Experiments
Search Spaces

Master parameter definitions, scaling strategies, and space sizing.

Search Spaces
Optimizers

Explore all 31 algorithms and learn when to use each one.

Optimizers