6.2.4.2. Optimizer Stoppers

A Stopper will cause an optimizer to stop and a new one to start (according to the OptimizerSelector). Stoppers can be used to define convergence criteria, or to stop optimizers that are performing poorly compared to the best optimizer.

Stoppers also follow the general pattern.

Tip

ExitConditions are different from Stoppers. To understand the difference between them, see the Difference Between Stoppers and ExitConditions.

By default, the current best optimizer is never stopped even if it satisfies the Stopper conditions, but this can be allowed with the ApplyStoppersToBestOptimizer option:

ApplyStoppersToBestOptimizer
Type:

Bool

Default value:

No

Description:

By default the stoppers are not applied to the best optimizer (the one who has seen the best value thus far). This is because many stoppers are based on comparisons to the best optimizer, and in most scenarios one would like to keep a well-performing optimizer alive. For some stopper configurations this paradigm does not make sense and we would prefer to apply the stoppers equally to all optimizers.

Stoppers are evaluated after every loss function call by default, but this interval can be adjusted with the CheckStopperInterval key:

CheckStopperInterval
Type:

Integer

Default value:

1

Description:

Number of loss function evaluations between evaluations of the stopper conditions.

You can specify multiple stopper conditions, and combine them using logical operators together with the StopperBooleanCombination option.

StopperBooleanCombination
Type:

String

GUI name:

Combine stoppers

Description:

If multiple Stoppers are used this is required to indicate how their evaluations relate to one another. Use an integer to refer to a stopper (defined by order in input file). Recognizes the symbols: ( ) & | E.g. (1 & 2) | 3. Defaults to an OR combination of all selected stoppers.

For example, to stop optimizers who have used more than 10000 function calls, or who have not improved their best value in a long time:

Stopper
    Type MaxFunctionCalls
    MaxFunctionCalls 10000
End
Stopper
    Type BestFunctionValueUnmoving
    BestFunctionValueUnmoving
        Tolerance 0.1
    End
End
StopperBooleanCombination 1 | 2

Available types of stoppers:

BestFunctionValueUnmoving

Stop optimizers that haven’t improved their best point in a long time.

CurrentFunctionValueUnmoving

Stop optimizers which have not changed their currently explored loss function value a lot in a long time.

MaxFunctionCalls

Stop optimizers when they have evaluated the loss function a certain number of times.

MaxInteroptimizerDistance

Stop optimizers too close together in parameter space.

MaxSequentialInvalidPoints

Stop optimizers that are stuck in areas of parameter space which return non-finite loss function values.

MinStepSize

Stop optimizers that are taking very small steps between iterations.

OptimizerType

For use with other conditions. Stop optimizers of a specific type.

TimeAnnealing

For use with other conditions. Keep an optimizer alive for longer if it’s relatively new.

ValidationWorsening

Stop optimizers which are evaluating points where the validation set loss value is worse than the average of several previously seen ones.

ValueAnnealing

For use with other conditions. Keep an optimizer alive if it looks likely to overtake the best optimizer.

Below we provide a technical, non-technical and graphical description for each of the available Stoppers so that you can understand their function.

6.2.4.2.1. BestFunctionValueUnmoving

This Stopper is designed to stop optimizers which are ‘lost’ and unable to improve the best loss value they have found. For example, CMA-ES will occasionally not show any degree of convergence over many evaluations when handling tough ReaxFF reparameterizations (see example below).

Intention

Input Block
“Stop optimizers which have not improved
by more than 10% in 900 function evaluations.”
 
Stopper
    Type BestFunctionValueUnmoving
    BestFunctionValueUnmoving
        NumberOfFunctionCalls 900
        Tolerance 0.1
    End
End
../../../_images/bestfunctionvalueunmoving.png

Fig. 6.1 Optimizer running loss after 1000 function evaluations at which point the Stopper is evaluated. Best points shown in red. “Best @ x” means the best loss value seen after x function evaluations.

Stopper
BestFunctionValueUnmoving
Type:

Block

Description:

Return True if an optimizer’s best value has not improved significantly in a given amount of time.

NumberOfFunctionCalls
Type:

Integer

Description:

Number of function evaluations between comparison points.

Tolerance
Type:

Float

Default value:

0.0

Description:

Function tolerance fraction between 0 and 1.

6.2.4.2.2. CurrentFunctionValueUnmoving

This Stopper stops optimizers which are fluctuating near a minimum but are unable to converge with their own internal conditions. This is a very common problem with ReaxFF reparameterizations; optimizers are unable to converge due to numerical instability near minima.

Intention

Input Block
“Stop optimizers which do not change the
function values they are exploring by
more than 10% in 400 function evaluations.”
 
Stopper
    Type CurrentFunctionValueUnmoving
    CurrentFunctionValueUnmoving
        NumberOfFunctionCalls 400
        Tolerance 0.1
    End
End
../../../_images/currentfunctionvalueunmoving.png

Fig. 6.2 Optimizer running loss after 1600 function evaluations at which point the Stopper is evaluated. Average and standard deviation shown in red.

Stopper
CurrentFunctionValueUnmoving
Type:

Block

GUI name:

Current function value unmoving:

Description:

Return True if an optimizer’s current function value has not improved significantly in a given amount of time.

NumberOfFunctionCalls
Type:

Integer

Description:

Number of function evaluations between comparison points.

Tolerance
Type:

Float

Default value:

0.0

Description:

Function tolerance fraction between 0 and 1.

6.2.4.2.3. MaxFunctionCalls

Simple Stopper to limit the number of function evaluations an optimizer is allowed. This is distinct from the similar ExitCondition MaxTotalFunctionCalls.

Intention

Input Block
“Stop an optimizer after it has
used 500 function evaluations.”
 
Stopper
    Type MaxFunctionCalls
    MaxFunctionCalls 500
End
../../../_images/maxfunctioncalls.png

Fig. 6.3 Two parallel optimizers, only optimizer 1 would be stopped by the MaxFunctionCalls Stopper. Evaluation numbers in ParAMS graphs are global, hence in this example the Stopper is only triggered after global evaluation 900, which corresponds to 500 calls for optimizer 1 and 400 calls for optimizer 2.

Stopper
MaxFunctionCalls
Type:

Integer

Description:

Returns True after an optimizer has evaluated at least n points. Use in an AND combination with other stoppers to keep optimizers alive for at least some period of time, or alone or in an OR combination to shut them down after some period of time.

6.2.4.2.4. MaxInteroptimizerDistance

Having two optimizer explore the same minimum is a waste of resources, particularly if you would like to identify multiple minima. This Stopper shuts down optimizers which are very close to one another in parameter space.

Intention

Input Block
“Stop an optimizer if it is within
5% of another one in parameter space.”
 
Stopper
    Type MaxInteroptimizerDistance
    MaxInteroptimizerDistance
        MaxRelativeDistance 0.05
        CompareAllOptimizers No
    End
End

In this example, we plot two optimizer trajectories in the 2D parameter space they are exploring. They both start at the same point, diverge in their initial exploration and then begin to move towards the same minimum. Out of the two, the optimizer that is stopped is the one that has the higher best loss function value.

In order to make the MaxInteroptimizerDistance easier to set up, the cutoff threshold is made relative to the longest distance in the space. This depends on the dimensionality of the problem, in most cases the parameters are scaled between 0 and 1 (as in the example below), and the longest distance in the space is \(\sqrt{n}\) where \(n\) is the number of optimized parameters.

CompareAllOptimizers determines which optimizers are compared by the Stopper. If No then the best optimizer (the one that has seen the lowest value) is compared to all other living optimizers. If Yes then all optimizer combinations are tested.

../../../_images/maxinteroptimizerdistance.png

Fig. 6.4 Two parallel optimizers, approaching the same minimum.

Stopper
MaxInteroptimizerDistance
Type:

Block

Description:

Return True if two optimizers are evaluating points too close to one another in parameter space.

CompareAllOptimizers
Type:

Bool

Default value:

No

Description:

If Yes the distance between all optimizer combinations are compared, otherwise worse optimizers are only compared to the best one.

MaxRelativeDistance
Type:

Float

Description:

Fraction (between 0 and 1) of the maximum distance in the space (from the point at all lower bounds to the point at all upper bounds) below which the optimizers are deemed too close and the victim will be stopped.

6.2.4.2.5. MaxSequentialInvalidPoints

Loss functions can return non-finite results if jobs are unable to converge or constraints are violated. Some optimizers may struggle to deal with non-finite values, or be unable to escape from that region of parameter space.

Intention

Input Block
“Stop optimizers which are stuck in a
non-finite region of parameter space
for more than 50 function evaluations.”
 
Stopper
    Type MaxSequentialInvalidPoints
    MaxSequentialInvalidPoints 50
End
../../../_images/maxsequentialinvalidpoints.png

Fig. 6.5 Optimizer evaluating many non-finite points (indicated by gaps in the graph). Stopper returns True at evaluation 150, the first time 50 sequential non-finite values occur. In ParAMS graphs only sequential finite points are connected, so there may be finite evaluations within the gaps.

Stopper
MaxSequentialInvalidPoints
Type:

Integer

Default value:

1

Description:

Return True if the optimizer has failed to find a finite function value in the last n iterations. Usually indicates a lost optimizer with poor starting point.

6.2.4.2.6. MinStepSize

Optimizers which are averaging very small step sizes should also be stopped. This is sometimes needed, instead of an internal optimizer step size conditions which typically look at single step sizes.

Intention

Input Block
“Stop an optimizer if it is
averaging steps of less than 5%
over the last 30 function evaluations.”
 
Stopper
    Type MinStepSize
    MinStepSize
        NumberOfFunctionCalls 30
        Tolerance 0.05
    End
End
../../../_images/minstepsize.png

Fig. 6.6 Optimizer trajectory in parameter space.

../../../_images/minstepsize2.png

Fig. 6.7 Optimizer step size taken as a function of function evaluation.

Stopper
MinStepSize
Type:

Block

Description:

Return True if an optimizer’s step size between evaluations is too small.

NumberOfFunctionCalls
Type:

Integer

Description:

Number of function evaluations between comparison points.

Tolerance
Type:

Float

Default value:

0.0

Description:

Fraction (between 0 and 1) of the maximum distance in the space (from the point at all lower bounds to the point at all upper bounds)

6.2.4.2.7. OptimizerType

This is an advanced Stopper intended for use in combination with others. Will be triggered if an optimizer is of a specific type. The intended use is to configure different stopping criteria for different optimizers if multiple types are used.

Intention

Input Block
“Stop CMA Optimizers.”
 
Stopper
    Type OptimizerType
    OptimizerType CMAES
End
Stopper
OptimizerType
Type:

Multiple Choice

Options:

[AdaptiveRateMonteCarlo, CMAES, Nevergrad, Scipy, RandomSampling, GridSampling]

Description:

Intended for use with other stoppers to allow for specific hunting conditions based on the type of optimizer.

6.2.4.2.8. TimeAnnealing

This is an advanced Stopper intended for use in combination with others. Optimizers are stopped randomly with some probability based on the time they have been alive. By using this Stopper with other conditions one can ensure optimizers are only stopped after they have been given some time to explore.

TimeAnnealing is resolved as follows:

\[\frac{n_b}{n_t} < u\]

where:

  • \(n_\text{b}\) is the number of function evaluations used by the best optimizer;

  • \(n_\text{t}\) is the number of function evaluations used by the optimizer we is considering stopping;

  • \(u\) is a uniformly randomly drawn test statistic (\(u ~ \mathscr{U}[0, u_\text{crit})\));

  • \(u_\text{crit}\) is the critical ratio of function evaluations below which a tested optimizer will not be stopped.

If the test is less than this number the optimizer is stopped.

Intention

Input Block
“Keep an optimizer alive at least until
it has evaluated half as many times as
the best optimizer.”
 
Stopper
    Type TimeAnnealing
    TimeAnnealing
        CriticalRatio 0.5
    End
End

The image below shows the probability of the Stopper being triggered as a function of the number of function evaluations used by the tested optimizer. In this example, the best optimizer (to which the tested optimizer is compared) has used 1000 function evaluations.

../../../_images/timeannealing.png

Fig. 6.8 Probability that the TimeAnnealing stopper will be triggered as a function of the number of function evaluations used by the tested optimizer. The best optimizer has used 1000 function evaluations.

Stopper
TimeAnnealing
Type:

Block

Description:

Keeps optimizers alive based on how long they have been alive. Randomly keeps optimizers alive based on how long (in function evaluations) they have been active. The newer an optimizer is, the more likely it will pass the test and be kept alive. Used to temper very strict termination conditions.

CriticalRatio
Type:

Float

Default value:

1.0

Description:

Critical ratio of function calls between stopper and victim above which the victim is guaranteed to survive. Values lower than one are looser and allow the victim to survive even if it has been in operation longer than the stopper. Values higher than one are stricter and may stop the victim even if it has iterated fewer times than the stopper.

6.2.4.2.9. ValidationWorsening

If a validation set is being used, test its loss function value and stop the optimizer if this value is getting worse. This is useful to guard against overfitting, a common problem during reparameterization.

Intention

Input Block
“Stop an optimizer if the validation set is
getting worse over the last 1000 evaluations.
Allow for fluctuations of 1%.”
 
Stopper
    Type ValidationUnmoving
    ValidationUnmoving
        NumberOfFunctionCalls 1000
        Tolerance 0.01
    End
End
../../../_images/validationunmoving.png

Fig. 6.9 Optimizer validation set running loss. Red line shows averaging.

Stopper
ValidationWorsening
Type:

Block

Description:

Return True if the loss value of the validation set is increasing.

NumberOfFunctionCalls
Type:

Integer

Default value:

1

Description:

Number of function evaluations between comparison points.

Tolerance
Type:

Float

Default value:

0.0

Description:

The loss value (f) is modified by: f * (1 + tol). This can also be used to ignore the effects of mild fluctuations.

6.2.4.2.10. ValueAnnealing

This is an advanced Stopper intended for use in combination with others. Optimizers are stopped randomly with some probability based on the best loss value they have seen. The intended use is to keep optimizers alive which otherwise meet the stopping conditions but are close to being the best optimizer, and thus have the potential to overtake it.

ValueAnnealing is resolved as follows:

\[\exp \left( s \left| \frac{f_b - f_t}{f_b} \right| \right) < u\]

where:

  • \(f_b\) is the lowest function value seen by the best optimizer;

  • \(f_t\) is the lowest loss function value seen by the optimizer the Stopper is considering stopping;

  • \(s\) is the critical stop chance defined as the probability of stopping the tested optimizer which is twice as large as the best optimizer in absolute value;

  • \(u\) is a uniformly randomly drawn test statistic such that \(u ~ \mathscr{U}[0, 1)\).

If the test is smaller than \(u\), the optimizer is stopped.

Intention

Input Block
“Keep an optimizer alive if it
has a reasonable chance of overtaking
the best optimizer.”
 
Stopper
    Type ValueAnnealing
    ValueAnnealing
        CriticalStopChance 0.5
    End
End
../../../_images/valueannealing.png

Fig. 6.10 Probability that the ValueAnnealing stopper will be triggered as a function of the loss value of the tested optimizer. The best optimizer’s best function value is \(10^4\) in this example.

Stopper
ValueAnnealing
Type:

Block

Description:

Intended for use with other stoppers. This condition is unlikely to stop a victim which has a loss function value very near the best seen thus far, but the probability of stopping increases with the difference between them.

CriticalStopChance
Type:

Float

Default value:

0.5

Description:

Value is the probability of stopping a victim which is twice as large as the stopper in absolute value. The default is 50%.