ParticleSwarmOptimizer#

class ParticleSwarmOptimizer(search_space=None, initialize=None, constraints=None, random_state=None, rand_rest_p=0.1, population=10, inertia=0.5, cognitive_weight=0.5, social_weight=0.5, temp_weight=0.2, n_iter=100, verbose=False, experiment=None)[source]#

Particle swarm optimizer.

Parameters:

search_spacedict[str, list]: The search space to explore. A dictionary with parameter names as keys and a numpy array as values.
initializedict[str, int]: The method to generate initial positions. A dictionary with the following key literals and the corresponding value type: {“grid”: int, “vertices”: int, “random”: int, “warm_start”: list[dict]}
constraintslist[callable]: A list of constraints, where each constraint is a callable. The callable returns True or False dependend on the input parameters.
random_stateNone, int: If None, create a new random state. If int, create a new random state seeded with the value.
rand_rest_pfloat: The probability of a random iteration during the the search process.
populationint: The number of particles in the swarm.
inertiafloat: The inertia of the swarm.
cognitive_weightfloat: A factor of the movement towards the personal best position of the individual optimizers in the population.
social_weightfloat: A factor of the movement towards the personal best position of the individual optimizers in the population.
temp_weightfloat: The temperature weight of the swarm.
n_iterint, default=100: The number of iterations to run the optimizer.
verbosebool, default=False: If True, print the progress of the optimization process.
experimentBaseExperiment, optional: The experiment to optimize parameters for. Optional, can be passed later via set_params.

Methods

`clone`()	Obtain a clone of the object with same hyper-parameters and config.
`clone_tags`(estimator[, tag_names])	Clone tags from another object as dynamic override.
`create_test_instance`([parameter_set])	Construct an instance of the class, using first test parameter set.
`create_test_instances_and_names`([parameter_set])	Create list of all test instances and a list of names for them.
`get_class_tag`(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
`get_class_tags`()	Get class tags from class, with tag level inheritance from parent classes.
`get_config`()	Get config flags for self.
`get_experiment`()	Get the experiment.
`get_param_defaults`()	Get object's parameter defaults.
`get_param_names`([sort])	Get object's parameter names.
`get_params`([deep])	Get a dict of parameters values for this object.
`get_search_config`()	Get the search configuration.
`get_tag`(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
`get_tags`()	Get tags from instance, with tag level inheritance and overrides.
`get_test_params`([parameter_set])	Get the test parameters for the optimizer.
`is_composite`()	Check if the object is composed of other BaseObjects.
`reset`()	Reset the object to a clean post-init state.
`set_config`(**config_dict)	Set config flags to given values.
`set_params`(**params)	Set the parameters of this object.
`set_random_state`([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
`set_tags`(**tag_dict)	Set instance level tag overrides to given values.
`solve`()	Run the optimization search process to maximize the experiment's score.

Examples

Basic usage of ParticleSwarmOptimizer with a scikit-learn experiment:

1. defining the experiment to optimize: >>> from hyperactive.experiment.integrations import SklearnCvExperiment >>> from sklearn.datasets import load_iris >>> from sklearn.svm import SVC >>> >>> X, y = load_iris(return_X_y=True) >>> >>> sklearn_exp = SklearnCvExperiment( … estimator=SVC(), … X=X, … y=y, … )

2. setting up the particleSwarmOptimizer optimizer: >>> from hyperactive.opt import ParticleSwarmOptimizer >>> import numpy as np >>> >>> config = { … “search_space”: { … “C”: [0.01, 0.1, 1, 10], … “gamma”: [0.0001, 0.01, 0.1, 1, 10], … }, … “n_iter”: 100, … } >>> optimizer = ParticleSwarmOptimizer(experiment=sklearn_exp, **config)

3. running the optimization: >>> best_params = optimizer.solve()

Best parameters can also be accessed via: >>> best_params = optimizer.best_params_

classmethod get_test_params(parameter_set='default')[source]#

Get the test parameters for the optimizer.

Returns:

dict with str keys: The test parameters dictionary.

clone()[source]#

Obtain a clone of the object with same hyper-parameters and config.

A clone is a different object without shared references, in post-init state. This function is equivalent to returning sklearn.clone of self.

Equivalent to constructing a new instance of type(self), with parameters of self, that is, type(self)(**self.get_params(deep=False)).

If configs were set on self, the clone will also have the same configs as the original, equivalent to calling cloned_self.set_config(**self.get_config()).

Also equivalent in value to a call of self.reset, with the exception that clone returns a new object, instead of mutating self like reset.

Raises:

RuntimeError if the clone is non-conforming, due to faulty __init__.

clone_tags(estimator, tag_names=None)[source]#

Clone tags from another object as dynamic override.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

clone_tags sets dynamic tag overrides from another object, estimator.

The clone_tags method should be called only in the __init__ method of an object, during construction, or directly after construction via __init__.

The dynamic tags are set to the values of the tags in estimator, with the names specified in tag_names.

The default of tag_names writes all tags from estimator to self.

Current tag values can be inspected by get_tags or get_tag.

Parameters:

estimatorAn instance of :class:BaseObject or derived class
tag_namesstr or list of str, default = None: Names of tags to clone. The default (None) clones all tags from estimator.

Returns:

self: Reference to self.

classmethod create_test_instance(parameter_set='default')[source]#

Construct an instance of the class, using first test parameter set.

Parameters:

parameter_setstr, default=”default”: Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.

Returns:

instanceinstance of the class with default parameters

classmethod create_test_instances_and_names(parameter_set='default')[source]#

Create list of all test instances and a list of names for them.

Parameters:

parameter_setstr, default=”default”: Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.

Returns:

objslist of instances of cls: i-th instance is cls(**cls.get_test_params()[i])
nameslist of str, same length as objs: i-th element is name of i-th instance of obj in tests. The naming convention is {cls.__name__}-{i} if more than one instance, otherwise {cls.__name__}

classmethod get_class_tag(tag_name, tag_value_default=None)[source]#

Get class tag value from class, with tag level inheritance from parents.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_class_tag method is a class method, and retrieves the value of a tag taking into account only class-level tag values and overrides.

It returns the value of the tag with name tag_name from the object, taking into account tag overrides, in the following order of descending priority:

Tags set in the _tags attribute of the class.
Tags set in the _tags attribute of parent classes,

in order of inheritance.

Does not take into account dynamic tag overrides on instances, set via set_tags or clone_tags, that are defined on instances.

To retrieve tag values with potential instance overrides, use the get_tag method instead.

Parameters:

tag_namestr: Name of tag value.
tag_value_defaultany type: Default/fallback value if tag is not found.

Returns:

tag_value: Value of the tag_name tag in self. If not found, returns tag_value_default.

classmethod get_class_tags()[source]#

Get class tags from class, with tag level inheritance from parent classes.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_class_tags method is a class method, and retrieves the value of a tag taking into account only class-level tag values and overrides.

It returns a dictionary with keys being keys of any attribute of _tags set in the class or any of its parent classes.

Values are the corresponding tag values, with overrides in the following order of descending priority:

Tags set in the _tags attribute of the class.
Tags set in the _tags attribute of parent classes,

in order of inheritance.

Instances can override these tags depending on hyper-parameters.

To retrieve tags with potential instance overrides, use the get_tags method instead.

Does not take into account dynamic tag overrides on instances, set via set_tags or clone_tags, that are defined on instances.

For including overrides from dynamic tags, use get_tags.

Returns:

collected_tagsdict: Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance. NOT overridden by dynamic tags set by set_tags or clone_tags.

get_config()[source]#

Get config flags for self.

Configs are key-value pairs of self, typically used as transient flags for controlling behaviour.

get_config returns dynamic configs, which override the default configs.

Default configs are set in the class attribute _config of the class or its parent classes, and are overridden by dynamic configs set via set_config.

Configs are retained under clone or reset calls.

Returns:

config_dictdict: Dictionary of config name : config value pairs. Collected from _config class attribute via nested inheritance and then any overrides and new tags from _onfig_dynamic object attribute.

get_experiment()[source]#

Get the experiment.

Returns:

BaseExperiment: The experiment to optimize parameters for.

classmethod get_param_defaults()[source]#

Get object’s parameter defaults.

Returns:

default_dict: dict[str, Any]: Keys are all parameters of cls that have a default defined in __init__. Values are the defaults, as defined in __init__.

classmethod get_param_names(sort=True)[source]#

Get object’s parameter names.

Parameters:

sortbool, default=True: Whether to return the parameter names sorted in alphabetical order (True), or in the order they appear in the class __init__ (False).

Returns:

param_names: list[str]: List of parameter names of cls. If sort=False, in same order as they appear in the class __init__. If sort=True, alphabetically ordered.

get_params(deep=True)[source]#

Get a dict of parameters values for this object.

Parameters:

deepbool, default=True

Whether to return parameters of components.

If True, will return a dict of parameter name : value for this object, including parameters of components (= BaseObject-valued parameters).
If False, will return a dict of parameter name : value for this object, but not include parameters of components.

Returns:

paramsdict with str-valued keys

Dictionary of parameters, paramname : paramvalue keys-value pairs include:

always: all parameters of this object, as via get_param_names values are parameter value for that key, of this object values are always identical to values passed at construction
if deep=True, also contains keys/value pairs of component parameters parameters of components are indexed as [componentname]__[paramname] all parameters of componentname appear as paramname with its value
if deep=True, also contains arbitrary levels of component recursion, e.g., [componentname]__[componentcomponentname]__[paramname], etc

get_search_config()[source]#

Get the search configuration.

Returns:

dict with str keys: The search configuration dictionary.

get_tag(tag_name, tag_value_default=None, raise_error=True)[source]#

Get tag value from instance, with tag level inheritance and overrides.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_tag method retrieves the value of a single tag with name tag_name from the instance, taking into account tag overrides, in the following order of descending priority:

Tags set via set_tags or clone_tags on the instance,

at construction of the instance.

Tags set in the _tags attribute of the class.
Tags set in the _tags attribute of parent classes,

in order of inheritance.

Parameters:

tag_namestr: Name of tag to be retrieved
tag_value_defaultany type, optional; default=None: Default/fallback value if tag is not found
raise_errorbool: whether a ValueError is raised when the tag is not found

Returns:

tag_valueAny: Value of the tag_name tag in self. If not found, raises an error if raise_error is True, otherwise it returns tag_value_default.

Raises:

ValueError, if raise_error is True.: The ValueError is then raised if tag_name is not in self.get_tags().keys().

get_tags()[source]#

Get tags from instance, with tag level inheritance and overrides.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_tags method returns a dictionary of tags, with keys being keys of any attribute of _tags set in the class or any of its parent classes, or tags set via set_tags or clone_tags.

Values are the corresponding tag values, with overrides in the following order of descending priority:

Tags set via set_tags or clone_tags on the instance,

at construction of the instance.

Tags set in the _tags attribute of the class.
Tags set in the _tags attribute of parent classes,

in order of inheritance.

Returns:

collected_tagsdict: Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance and then any overrides and new tags from _tags_dynamic object attribute.

is_composite()[source]#

Check if the object is composed of other BaseObjects.

A composite object is an object which contains objects, as parameters. Called on an instance, since this may differ by instance.

Returns:

composite: bool: Whether an object has any parameters whose values are BaseObject descendant instances.

reset()[source]#

Reset the object to a clean post-init state.

Results in setting self to the state it had directly after the constructor call, with the same hyper-parameters. Config values set by set_config are also retained.

A reset call deletes any object attributes, except:

hyper-parameters = arguments of __init__ written to self, e.g., self.paramname where paramname is an argument of __init__
object attributes containing double-underscores, i.e., the string “__”. For instance, an attribute named “__myattr” is retained.
config attributes, configs are retained without change. That is, results of get_config before and after reset are equal.

Class and object methods, and class attributes are also unaffected.

Equivalent to clone, with the exception that reset mutates self instead of returning a new object.

After a self.reset() call, self is equal in value and state, to the object obtained after a constructor call``type(self)(**self.get_params(deep=False))``.

Returns:

self: Instance of class reset to a clean post-init state but retaining the current hyper-parameter values.

set_config(**config_dict)[source]#

Set config flags to given values.

Configs are key-value pairs of self, typically used as transient flags for controlling behaviour.

set_config sets dynamic configs, which override the default configs.

Default configs are set in the class attribute _config of the class or its parent classes, and are overridden by dynamic configs set via set_config.

Configs are retained under clone or reset calls.

Parameters:

config_dictdict: Dictionary of config name : config value pairs.

Returns:

selfreference to self.

Notes

Changes object state, copies configs in config_dict to self._config_dynamic.

set_params(**params)[source]#

Set the parameters of this object.

The method works on simple skbase objects as well as on composite objects. Parameter key strings <component>__<parameter> can be used for composites, i.e., objects that contain other objects, to access <parameter> in the component <component>. The string <parameter>, without <component>__, can also be used if this makes the reference unambiguous, e.g., there are no two parameters of components with the name <parameter>.

Parameters:

**paramsdict: BaseObject parameters, keys must be <component>__<parameter> strings. __ suffixes can alias full strings, if unique among get_params keys.

Returns:

selfreference to self (after parameters have been set)

set_random_state(random_state=None, deep=True, self_policy='copy')[source]#

Set random_state pseudo-random seed parameters for self.

Finds random_state named parameters via self.get_params, and sets them to integers derived from random_state via set_params. These integers are sampled from chain hashing via sample_dependent_seed, and guarantee pseudo-random independence of seeded random generators.

Applies to random_state parameters in self, depending on self_policy, and remaining component objects if and only if deep=True.

Note: calls set_params even if self does not have a random_state, or none of the components have a random_state parameter. Therefore, set_random_state will reset any scikit-base object, even those without a random_state parameter.

Parameters:

random_stateint, RandomState instance or None, default=None

Pseudo-random number generator to control the generation of the random integers. Pass int for reproducible output across multiple function calls.

deepbool, default=True

Whether to set the random state in skbase object valued parameters, i.e., component estimators.

If False, will set only self’s random_state parameter, if exists.
If True, will set random_state parameters in component objects as well.

self_policystr, one of {“copy”, “keep”, “new”}, default=”copy”

“copy” : self.random_state is set to input random_state
“keep” : self.random_state is kept as is
“new” : self.random_state is set to a new random state,

derived from input random_state, and in general different from it

Returns:

selfreference to self

set_tags(**tag_dict)[source]#

Set instance level tag overrides to given values.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

set_tags sets dynamic tag overrides to the values as specified in tag_dict, with keys being the tag name, and dict values being the value to set the tag to.

The set_tags method should be called only in the __init__ method of an object, during construction, or directly after construction via __init__.

Current tag values can be inspected by get_tags or get_tag.

Parameters:

**tag_dictdict: Dictionary of tag name: tag value pairs.

Returns:

Self: Reference to self.

solve()[source]#

Run the optimization search process to maximize the experiment’s score.

The optimization searches for a maximizer of the experiment’s score method.

Depending on the tag property:higher_or_lower_is_better being set to higher or lower, the run method will search for:

the minimizer of the evaluate method if the tag is lower
the maximizer of the evaluate method if the tag is higher

Returns:

best_paramsdict: The best parameters found during the optimization process. The dict best_params can be used in experiment.score or experiment.evaluate directly.