Introduction

Optimization finds the best parameters from a set of possibilities. Evaluating every combination is usually impractical, so algorithms guide the search. Hyperactive provides a unified interface: define your problem once, then swap between different optimization algorithms without changing your code.

---

Why Hyperactive?

Hyperactive makes optimization simple. Define your problem once, then swap between 31 different algorithms with a single line change.

Swap Algorithms

Try different optimizers without rewriting code. One line change switches from hill climbing to Bayesian optimization.

31 Algorithms

Local search, global search, population-based, and model-based methods. All with the same simple interface.

ML Integrations

Ready-to-use experiments for sklearn, sktime, skpro, and PyTorch. Tune models with minimal code.

Any Python Function

Works with simulations, engineering problems, or any function that returns a score.

---

Quick Start

The simplest optimization in 5 lines:

from hyperactive.opt.gfo import HillClimbing

def score(p):
    return -(p["x"] ** 2)  # Find x that minimizes x²

opt = HillClimbing({"x": range(-10, 11)}, experiment=score)
best = opt.solve()  # {"x": 0}

---

ML Integration Examples

Tune machine learning models with ready-to-use experiment classes:

sklearn

from hyperactive.experiment.integrations import SklearnCvExperiment
from sklearn.ensemble import GradientBoostingClassifier

experiment = SklearnCvExperiment(GradientBoostingClassifier(), X, y, cv=5)

sktime

from hyperactive.experiment.integrations import SktimeForecastingExperiment
from sktime.forecasting.arima import ARIMA

experiment = SktimeForecastingExperiment(ARIMA(), y_train, fh=[1, 2, 3])

PyTorch

from hyperactive.experiment.integrations import TorchTrainerExperiment
import pytorch_lightning as pl

experiment = TorchTrainerExperiment(YourLightningModule, train_loader)

See Framework Integrations for complete examples and all available integrations.

---

Core Concepts

Hyperactive is built around three simple concepts:

What to optimize

Experiment

Any Python function that takes parameters and returns a score. Hyperactive will maximize this score.

def experiment(params):
    return score

Learn more

Where to search

Search Space

A dictionary mapping parameter names to possible values. Defines the boundaries of your optimization.

{"x": [1, 2, 3], "y": ["a", "b"]}

Learn more

How to search

Optimizer

The algorithm that explores the search space. Choose based on your problem characteristics.

HillClimbing(space, experiment=exp)

Learn more

---

The Power of Swapping

Define your problem once, then try different algorithms with a single line change:

Hill Climbing

from hyperactive.opt.gfo import HillClimbing

optimizer = HillClimbing(search_space, experiment=experiment)
best = optimizer.solve()

Fast local search. Good for quick exploration.

Bayesian Optimization

from hyperactive.opt.gfo import BayesianOptimizer

optimizer = BayesianOptimizer(search_space, experiment=experiment)
best = optimizer.solve()

Learns from past evaluations. Best for expensive functions.

Genetic Algorithm

from hyperactive.opt.gfo import GeneticAlgorithmOptimizer

optimizer = GeneticAlgorithmOptimizer(search_space, experiment=experiment)
best = optimizer.solve()

Population-based evolution. Great for complex landscapes.

Tip

The experiment and search space stay the same. Only the optimizer changes. This makes it easy to benchmark different algorithms on your problem.

---

Complete Example

Here’s a full working example that tunes a Random Forest classifier:

import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from hyperactive.experiment.integrations import SklearnCvExperiment
from hyperactive.opt.gfo import BayesianOptimizer

# 1. Load your data
X, y = load_iris(return_X_y=True)

# 2. Define the experiment (what to optimize)
experiment = SklearnCvExperiment(
    estimator=RandomForestClassifier(),
    X=X, y=y, cv=5,
)

# 3. Define the search space (where to search)
search_space = {
    "n_estimators": list(range(10, 200, 10)),
    "max_depth": [3, 5, 10, 20, None],
    "min_samples_split": [2, 5, 10],
}

# 4. Choose an optimizer (how to search)
optimizer = BayesianOptimizer(
    search_space=search_space,
    n_iter=50,
    experiment=experiment,
    random_state=42,
)

# 5. Run and get the best parameters
best_params = optimizer.solve()
print(f"Best parameters: {best_params}")

---

Common Parameters

All optimizers share these parameters:

Parameter	Type	Description
search_space	dict	Maps parameter names to possible values
n_iter	int	Number of optimization iterations
experiment	callable	The objective function or experiment object
random_state	int	Seed for reproducibility
initialize	dict	Control initial population (warm starts, etc.)

Warm Starting

You can provide starting points for optimization:

warm_start = [
    {"n_estimators": 100, "max_depth": 10},  # Start from known good point
]

optimizer = HillClimbing(
    search_space=search_space,
    n_iter=50,
    experiment=experiment,
    initialize={"warm_start": warm_start},
)

---

Tips for Beginners

Start Simple

Begin with HillClimbing or RandomSearch to establish baselines before trying sophisticated algorithms.

Right-Size Your Space

Large search spaces need more iterations. Use np.logspace for parameters that span orders of magnitude.

Set Random State

For reproducible results, always set random_state=42 (or any integer).

Match Algorithm to Budget

Expensive evaluations? Use BayesianOptimizer which learns from each evaluation. Cheap evaluations? RandomSearch explores well.

---

Next Steps

Experiments

Learn how to define what to optimize, including custom functions and built-in ML experiments.

Experiments

Search Spaces

Master parameter definitions, scaling strategies, and space sizing.

Search Spaces

Optimizers

Explore all 31 algorithms and learn when to use each one.

Optimizers