6.2.5. Generator, Selector, Spawning¶
6.2.5.1. Initial parameters generator¶
When a new optimizer is started, the initial parameters can be controlled with the Generator block which follows the general pattern:
Generator
ExploreExploit
Focus float
MaxFunctionCalls integer
End
Perturbation
StandardDeviation float
End
Type [Incumbent | ExploreExploit | Perturbation | Random | SinglePoint]
End
Generator- Type
Block
- GUI name
Starting point generator:
- Description
A Generator used to produce x0 starting points for the optimizers.
ExploreExploit- Type
Block
- Description
Blends a randomly generated point with the location of an existing optimizer, based on the time progress through the optimization. The optimizer is chosen based on a weighted roulette selection based on their function value. Early in the optimization optimizers are started randomly, and later they are started near previously found good minima.
Focus- Type
Float
- Default value
1.0
- Description
The blend parameter between random point and incumbent points. Used as follows: p=(f_calls / max_f_calls) ** focus
MaxFunctionCalls- Type
Integer
- Description
Maximum function calls allowed for the optimization, at and beyond this point there is a 100% chance that a previously evaluated point will be returned by the generator. If the optimization is not limited by the number of function calls, provide an estimate.
Perturbation- Type
Block
- Description
Randomly generates parameter vectors from a multivariate normal distribution around the starting parameters.
StandardDeviation- Type
Float
- Default value
0.2
- Description
Standard deviation of the multivariate normal distribution. Used here to control how wide the generator should explore around the starting parameters.
Type- Type
Multiple Choice
- Default value
SinglePoint
- Options
[Incumbent, ExploreExploit, Perturbation, Random, SinglePoint]
- GUI name
Starting points generator
- Description
Algorithm used to pick starting points for the optimizers. Available options: • Incumbent: Optimizers will be started at the best point seen thus far by any optimizer. First point is random. • ExploreExploit: Early starting points are random, but later points are closer to good minima. • Perturbation: Each starting point is a drawn from a multivariate Gaussian distribution centred at the initial parameter set. • Random: Optimizers are started at random locations in parameter space. • SinglePoint: All optimizers are started at the initial parameter values.
6.2.5.2. Optimizer selector¶
If you have multiple Optimizer blocks, then when a new Optimizer is started
it will be selected according to the rule defined by the OptimizerSelector which follows the general pattern:
OptimizerSelector
Chain
Thresholds integer_list
End
Type [Cycle | Random | Chain]
End
OptimizerSelector- Type
Block
- GUI name
Selection between optimizer types:
- Description
If multiple Optimizers are included, then this block must be included and configures the Selector which will choose between them.
Chain- Type
Block
- Description
Start different optimizers at different points in time based on the number of function evaluations used.
Thresholds- Type
Integer List
- Description
List of loss function evaluation thresholds which switch to the next optimizer in the list. If there are n optimizers available, then this list should have n-1 thresholds.
Type- Type
Multiple Choice
- Default value
Cycle
- Options
[Cycle, Random, Chain]
- GUI name
Select optimizer by
- Description
Name of the algorithm selecting between optimizers. Available options: • Cycle (Default): Sequential loop through available optimizers. • Random: Optimizers are selected randomly from the available options. • Chain: Time-based progression through the list of available optimizers. The next option will be started once a certain number of loss function evaluations have been used.
6.2.5.3. Control optimizers spawning¶
ControlOptimizerSpawning
MaxEvaluations integer
MaxOptimizers integer
End
ControlOptimizerSpawning- Type
Block
- Description
Control the spawning of optimizers. Note, this is different from ExitConditions. These simply stop new optimizers from spawning, but leave existing ones untouched. ExitConditions shutdown active optimizers and stop the optimization.
MaxEvaluations- Type
Integer
- GUI name
– n loss function evaluations
- Description
No new optimizers will be started after this number of function evaluations has been used. Note, this is different from the equivalent exit condition which would terminate existing optimizers rather than simply not allowing new ones to spawn.
MaxOptimizers- Type
Integer
- GUI name
– n optimizers started
- Description
No new optimizers will be started after this number has been spawned. Note, this is different from the equivalent exit condition which would terminate existing optimizers rather than simply not allowing new ones to spawn.