Sequential Model-Based Algorithms

Sequential model-based optimization (SMBO) algorithms build a surrogate model of the objective function to guide the search. They are particularly effective when function evaluations are expensive.

Algorithm Examples

Algorithm	Example
Bayesian Optimization	bayesian_optimization_example.py
Tree-Parzen Estimators	tree_structured_parzen_estimators_example.py
Forest Optimizer	forest_optimizer_example.py
Lipschitz Optimizer	lipschitz_optimizer_example.py
DIRECT Algorithm	direct_algorithm_example.py

When to Use Model-Based Methods

Sequential model-based algorithms are best suited for:

• Expensive objective functions (e.g., training neural networks, simulations)

• Limited evaluation budgets where each evaluation counts

• Smooth, continuous search spaces where surrogate models work well

• Hyperparameter optimization for machine learning models

See Optimizers for detailed algorithm descriptions.