Optimizers#

Hyperactive provides 31 algorithms across 5 categories and 3 backends. Optimizers navigate the search space to find optimal parameters. Each implements a different strategy for balancing exploration (trying diverse regions) and exploitation (refining promising solutions). Local search methods like Hill Climbing work well for smooth landscapes. Population-based methods handle multiple local optima. Model-based methods like Bayesian Optimization minimize evaluations for expensive objective functions.

Algorithm Landscape#

Hyperactive optimizer taxonomy showing 31 algorithms across GFO, Optuna, and sklearn backends

Quick Selection Guide#

Not sure which optimizer to use? Start here:

Quick baseline

Use: RandomSearch or HillClimbing

Fast, simple, good starting point for any problem.

Expensive evaluations

Use: BayesianOptimizer or TPEOptimizer

Learn from past evaluations, minimize function calls.

Many local optima

Use: GeneticAlgorithm or DifferentialEvolution

Population-based methods escape local traps.

Small search space

Use: GridSearch

Exhaustive coverage when feasible (<1000 combinations).

Tip

Always run RandomSearch first to establish a baseline and understand your objective landscape before trying more sophisticated algorithms.

Algorithm Categories#

5 algorithms

Local Search

Explore neighborhoods of current solutions. Fast but may get stuck in local optima.

HillClimbing, SimulatedAnnealing, StochasticHillClimbing, RepulsingHillClimbing, DownhillSimplex

Local Search

4 algorithms

Global Search

Explore the entire search space systematically or randomly.

RandomSearch, GridSearch, RandomRestartHillClimbing, Powell’s Method

Global Search

6 algorithms

Population-Based

Evolve multiple candidate solutions together. Excellent for complex landscapes.

ParticleSwarm, GeneticAlgorithm, EvolutionStrategy, DifferentialEvolution, ParallelTempering, SpiralOptimization

Population Methods

5 algorithms

Model-Based (SMBO)

Build surrogate models to guide search. Best for expensive evaluations.

BayesianOptimizer, TPE, ForestOptimizer, Lipschitz, DIRECT

Sequential Model-Based Methods

8 algorithms

Optuna Backend

Wrappers for Optuna’s powerful samplers with Hyperactive’s interface.

TPEOptimizer, CmaEsOptimizer, GPOptimizer, NSGAIIOptimizer, and more

Optuna Backend

Scenario Reference#

Detailed recommendations based on problem characteristics:

Scenario	Recommended Optimizers	Why
Quick baseline	`HillClimbing`, `RandomSearch`	Fast, simple, good for initial exploration
Expensive evaluations	`BayesianOptimizer`, `TPEOptimizer`	Learn from past evaluations, minimize function calls
Large search space	`RandomSearch`, `ParticleSwarmOptimizer`	Good global coverage without exhaustive search
Multi-modal landscape	`GeneticAlgorithm`, `DifferentialEvolution`	Population-based methods avoid local optima
Small search space	`GridSearch`	Exhaustive coverage when feasible
Continuous parameters	`BayesianOptimizer`, `CmaEsOptimizer`	Designed for smooth, continuous spaces
Mixed parameter types	`TPEOptimizer`, `RandomSearch`	Handle categorical + continuous well

Configuration#

All optimizers share common parameters and configuration options.