6.2.5. Generator, Selector, Spawning¶

6.2.5.1. Initial parameters generator¶

When a new optimizer is started, the initial parameters can be controlled with the Generator block which follows the general pattern:

Generator
   ExploreExploit
      Focus float
      MaxFunctionCalls integer
   End
   Perturbation
      StandardDeviation float
   End
   Type [Incumbent | ExploreExploit | Perturbation | Random | SinglePoint]
End

Generator

Type:

Block

GUI name:

Starting point generator:

Description:

A Generator used to produce x0 starting points for the optimizers.

ExploreExploit

Type:

Block

Description:

Blends a randomly generated point with the location of an existing optimizer, based on the time progress through the optimization. The optimizer is chosen based on a weighted roulette selection based on their function value. Early in the optimization optimizers are started randomly, and later they are started near previously found good minima.

Focus

Type:

Float

Default value:

1.0

Description:

The blend parameter between random point and incumbent points. Used as follows: p=(f_calls / max_f_calls) ** focus

MaxFunctionCalls

Type:

Integer

Description:

Maximum function calls allowed for the optimization, at and beyond this point there is a 100% chance that a previously evaluated point will be returned by the generator. If the optimization is not limited by the number of function calls, provide an estimate.

Perturbation

Type:

Block

Description:

Randomly generates parameter vectors from a multivariate normal distribution around the starting parameters.

StandardDeviation

Type:

Float

Default value:

0.2

Description:

Standard deviation of the multivariate normal distribution. Used here to control how wide the generator should explore around the starting parameters.

Type

Type:

Multiple Choice

Default value:

SinglePoint

Options:

[Incumbent, ExploreExploit, Perturbation, Random, SinglePoint]

GUI name:

Starting points generator

Description:

Algorithm used to pick starting points for the optimizers. Available options: • Incumbent: Optimizers will be started at the best point seen thus far by any optimizer. First point is random. • ExploreExploit: Early starting points are random, but later points are closer to good minima. • Perturbation: Each starting point is a drawn from a multivariate Gaussian distribution centred at the initial parameter set. • Random: Optimizers are started at random locations in parameter space. • SinglePoint: All optimizers are started at the initial parameter values.

6.2.5.2. Optimizer selector¶

If you have multiple Optimizer blocks, then when a new Optimizer is started it will be selected according to the rule defined by the OptimizerSelector which follows the general pattern:

OptimizerSelector
   Chain
      Thresholds integer_list
   End
   Type [Cycle | Random | Chain]
End

OptimizerSelector

Type:

Block

GUI name:

Selection between optimizer types:

Description:

If multiple Optimizers are included, then this block must be included and configures the Selector which will choose between them.

Chain

Type:

Block

Description:

Start different optimizers at different points in time based on the number of function evaluations used.

Thresholds

Type:

Integer List

Description:

List of loss function evaluation thresholds which switch to the next optimizer in the list. If there are n optimizers available, then this list should have n-1 thresholds.

Type

Type:

Multiple Choice

Default value:

Cycle

Options:

[Cycle, Random, Chain]

GUI name:

Select optimizer by

Description:

Name of the algorithm selecting between optimizers. Available options: • Cycle (Default): Sequential loop through available optimizers. • Random: Optimizers are selected randomly from the available options. • Chain: Time-based progression through the list of available optimizers. The next option will be started once a certain number of loss function evaluations have been used.

6.2.5.3. Control optimizers spawning¶

ControlOptimizerSpawning
   MaxEvaluations integer
   MaxOptimizers integer
End

ControlOptimizerSpawning

Type:

Block

Description:

Control the spawning of optimizers. Note, this is different from ExitConditions. These simply stop new optimizers from spawning, but leave existing ones untouched. ExitConditions shutdown active optimizers and stop the optimization.

MaxEvaluations

Type:

Integer

GUI name:

– n loss function evaluations

Description:

No new optimizers will be started after this number of function evaluations has been used. Note, this is different from the equivalent exit condition which would terminate existing optimizers rather than simply not allowing new ones to spawn.

MaxOptimizers

Type:

Integer

GUI name:

– n optimizers started

Description:

No new optimizers will be started after this number has been spawned. Note, this is different from the equivalent exit condition which would terminate existing optimizers rather than simply not allowing new ones to spawn.