Active Learning with uncertainties predicted from committee models¶
Trained model: A committee model consisting of 2 custom M3GNet models, starting from random weight initialization
Reference method: DFT, PBE-D3(BJ), engine ADF
System: H₂O + propene → 1-propanol
Problem: M3GNet-UP-2022 underestimates the reaction barrier
Solution: Retraining the model gives better agreement:
Expected duration: This notebook takes about 48 hours to run on the CPU. This includes both the active learning and the various NEB calculations.
Approach: In addition to dividing an MD simulation into several active learning steps, we will also exploit the uncertainty that ML models can give when trained as committee models. By setting up a ReactionBoost MD simulation to force the desired reaction and applying uncertainty-based exit conditions, we effectively choose training points where the model is “uncertain”.
Key takeaways:
Training M3GNet models from scratch typically requires more training data than starting from the Universal Potential
ReactionBoost can be effectively used to sample structures during a chemical reaction but may not sample the minimum energy path.
In particular for models trained from scratch like this, to get a good result for the reaction barrier we also need to create a separate “mini-active-learning-loop” in Python that runs NEB, recalculates with DFT, retrains the model, runs NEB, etc., a few times before we get a good result also for NEB calculations.
Technical note: This example uses the new “Python Input System for AMS” input classes instead of the normal PLAMS settings object. Both methods work.
To follow along, either
Download
sal_uncertainty.py(run as$AMSBIN/amsipython sal_uncertainty.py).Download
sal_uncertainty.ipynb(see also: how to install Jupyterlab in AMS)
Initial imports¶
import scm.plams as plams
from scm.simple_active_learning import SimpleActiveLearningJob
from scm.reactmap.tools import reorder_plams_mol
import matplotlib.pyplot as plt
from scm.input_classes import drivers, engines
from scm.libbase import Units
import scm.params as params
import os
import numpy as np
plams.init()
PLAMS working folder: /path/plams_workdir.002
system1 = plams.from_smiles("CC=C", forcefield="uff") # propene
# add a water molecule *at least* 1 angstrom away from all propene atoms
system1.add_molecule(plams.from_smiles("O"), margin=1)
for at in system1:
at.properties = plams.Settings()
system1.delete_all_bonds()
plams.plot_molecule(system1)
plt.title("System 1: propene + H2O");
system2 = plams.from_smiles("CCCO") # 1-propanol
for at in system2:
at.properties = plams.Settings()
system2.delete_all_bonds()
# reorder atoms in system2 to match the order in system1
# this only takes bond breaking and forming into account, the order is not guaranteed to match exactly for all atoms
system2 = reorder_plams_mol(system1, system2)
# Rotate system2 so that the RMSD with respect to system1 is minimized
system2.align2mol(system1)
plams.plot_molecule(system2)
plt.title("System 2: 1-propanol");
# sanity-check that at least the order of elements is identical
assert list(system1.symbols) == list(system2.symbols), f"Something went wrong!"
Note that this does not guarantee that the atom order is completely the same. For example the order of the hydrogen atoms in the CH3 group might be different. This means that we cannot just run NEB directly. So let’s first run MD ReactionBoost.
Initial Reaction Boost to get reactant and product¶
Engine settings¶
Here we use e_up to refer to the M3GNet Universal Potential.
For the ADF DFT engine we set an electronic temperature and the OptimizeSpinRound option. This helps with SCF convergence, and can converge the SCF to a different spin state when applicable.
e_up = engines.MLPotential()
e_up.Model = "M3GNet-UP-2022"
e_dft = engines.ADF()
e_dft.XC.GGA = "PBE"
e_dft.XC.Dispersion = "GRIMME3 BJDAMP"
e_dft.Basis.Type = "TZP"
e_dft.Unrestricted = True
e_dft.Occupations = "ElectronicTemperature=300 OptimizeSpinRound=0.05"
def set_reaction_boost(driver, nsteps=3000):
driver.Task = "MolecularDynamics"
md = driver.MolecularDynamics
md.InitialVelocities.Temperature = 100
md.NSteps = nsteps
md.ReactionBoost.Type = "Pair"
md.ReactionBoost.BondBreakingRestraints.Type = "Erf"
md.ReactionBoost.BondBreakingRestraints.Erf.MaxForce = 0.05
md.ReactionBoost.BondMakingRestraints.Type = "Erf"
md.ReactionBoost.BondMakingRestraints.Erf.MaxForce = 0.12
md.ReactionBoost.InitialFraction = 0.05
md.ReactionBoost.Change = "LogForce"
md.ReactionBoost.NSteps = nsteps
md.ReactionBoost.TargetSystem[0] = "final"
md.Trajectory.SamplingFreq = 10
md.Trajectory.WriteBonds = False
md.Trajectory.WriteMolecules = False
md.TimeStep = 0.25
md.Thermostat[0].Tau = 5
md.Thermostat[0].Temperature = [100.0]
md.Thermostat[0].Type = "Berendsen"
def get_reaction_boost_job(engine, molecule, name: str = "reaction_boost") -> plams.AMSJob:
d = drivers.AMS()
set_reaction_boost(d)
d.Engine = engine
job = plams.AMSJob(settings=d, name=name, molecule=molecule)
job.settings.runscript.nproc = 1
return job
molecule_dict = {"": system1, "final": system2}
prelim_job = get_reaction_boost_job(e_up, molecule_dict, "prelim_md")
prelim_job.run();
[03.04|14:11:30] JOB prelim_md STARTED
[03.04|14:11:30] JOB prelim_md RUNNING
[03.04|14:12:19] JOB prelim_md FINISHED
[03.04|14:12:19] JOB prelim_md SUCCESSFUL
Let’s check that the final molecule corresponds to the target system (1-propanol):
engine_energies = prelim_job.results.get_history_property("EngineEnergy")
N_frames = len(engine_energies)
max_index = np.argmax(engine_energies)
reactant_index = max(0, max_index - 50) # zero-based
system1_correct_order = prelim_job.results.get_history_molecule(reactant_index + 1)
system1_correct_order.delete_all_bonds()
plams.plot_molecule(system1_correct_order)
product_index = min(max_index + 50, N_frames - 1) # zero-based
system2_correct_order = prelim_job.results.get_history_molecule(product_index + 1)
system1_correct_order.delete_all_bonds()
plams.plot_molecule(system2_correct_order)
We now have the product molecule with the correct atom order, which means we can run an initial NEB with M3GNet and compare to the DFT reference:
Initial NEB calculation¶
molecule_dict = {"": system1_correct_order, "final": system2_correct_order}
def get_neb_job(engine, name: str = "neb") -> plams.AMSJob:
d = drivers.AMS()
d.Task = "NEB"
d.GeometryOptimization.Convergence.Quality = "Basic"
d.NEB.Images = 12
d.Engine = engine
neb_job = plams.AMSJob(name=name, settings=d, molecule=molecule_dict)
return neb_job
neb_job = get_neb_job(e_up, name="neb_up")
neb_job.run();
[03.04|14:14:19] JOB neb_up STARTED
[03.04|14:14:19] JOB neb_up RUNNING
[03.04|14:14:45] JOB neb_up FINISHED
[03.04|14:14:46] JOB neb_up SUCCESSFUL
Let’s then replay with the ADF DFT engine.
def get_replay_job(rkf, name="replay"):
d_replay = drivers.AMS()
d_replay.Task = "Replay"
d_replay.Replay.File = os.path.abspath(rkf)
d_replay.Properties.Gradients = True
d_replay.Engine = e_dft
replay_job = plams.AMSJob(name=name, settings=d_replay)
return replay_job
replay_job = get_replay_job(neb_job.results.rkfpath(), "replay_neb")
replay_job.run();
[03.04|14:14:51] JOB replay_neb STARTED
[03.04|14:14:51] JOB replay_neb RUNNING
[03.04|14:15:57] JOB replay_neb FINISHED
[03.04|14:15:58] JOB replay_neb SUCCESSFUL
def get_relative_energies(neb_job):
e = neb_job.results.get_neb_results()["Energies"]
e = np.array(e) - np.min(e)
e *= Units.get_factor("hartree", "eV")
return e
def plot_neb_comparison(neb_job, replay_job, legend=None, title=None):
energies_up = get_relative_energies(neb_job)
energies_dft = get_relative_energies(replay_job)
fig, ax = plt.subplots()
ax.plot(energies_up)
ax.plot(energies_dft)
ax.legend(legend or ["M3GNet-UP-2022", "DFT singlepoints"])
ax.set_ylabel("Relative energy (eV)")
ax.set_title(title or "Reaction path water+propene -> 1-propanol")
return ax
plot_neb_comparison(neb_job, replay_job);
So we can see that either M3GNet-UP-2022 underestimates the barrier or it NEB path is different from the DFT one. Let’s use these datapoints as a starting point for the active learning.
Let’s also run replay on some of the frames from the prelim_md reaction boost job:
replay_md = get_replay_job(prelim_job.results.rkfpath(), "replay_md")
N_frames_to_replay = 10
replay_md.settings.input.Replay.Frames = list(
np.linspace(reactant_index, product_index, N_frames_to_replay, dtype=np.int64)
)
replay_md.run();
[03.04|14:16:22] JOB replay_md STARTED
[03.04|14:16:22] JOB replay_md RUNNING
[03.04|14:17:11] JOB replay_md FINISHED
[03.04|14:17:12] JOB replay_md SUCCESSFUL
Simple Active Learning using Uncertainties¶
Create the initial reference data¶
yaml_dir = "my_neb_data"
ri = params.ResultsImporter.from_ase() # use ASE units
ri.add_trajectory_singlepoints(
replay_job.results.rkfpath(), properties=["energy", "forces"], data_set="training_set"
)
ri.add_trajectory_singlepoints(
replay_md.results.rkfpath(),
properties=["energy", "forces"],
data_set="training_set",
indices=list(range(1, N_frames_to_replay - 1)),
)
ri.add_trajectory_singlepoints(
replay_md.results.rkfpath(),
properties=["energy", "forces"],
indices=[0, N_frames_to_replay - 1],
data_set="validation_set",
)
ri.store(yaml_dir, backup=False)
['my_neb_data/job_collection.yaml',
'my_neb_data/results_importer_settings.yaml',
'my_neb_data/training_set.yaml',
'my_neb_data/validation_set.yaml']
When we have initial reference data like this, it’s often most convenient to run a separate ParAMS training before starting the active learning.
This lets us sanity-check the training parameters, and more easily try different Active Learning settings without having to retrain the initial model every time.
def get_params_job(yaml_dir, load_model=None, name="paramsjob"):
committee_size = 2
paramsjob = params.ParAMSJob.from_yaml(yaml_dir, use_relative_paths=True, name=name)
paramsjob.settings.input.Task = "MachineLearning"
ml = paramsjob.settings.input.MachineLearning
ml.Backend = "M3GNet"
if load_model:
ml.LoadModel = load_model
ml.MaxEpochs = 200
ml.M3GNet.LearningRate = 5e-4
else:
ml.M3GNet.Model = "Custom"
ml.M3GNet.Custom.NumNeurons = 32
ml.MaxEpochs = 300
ml.M3GNet.LearningRate = 1e-3
ml.CommitteeSize = committee_size
paramsjob.settings.input.ParallelLevels.CommitteeMembers = committee_size
return paramsjob
paramsjob = get_params_job(yaml_dir, name="custom_initial_training")
paramsjob.run();
[03.04|14:17:48] JOB custom_initial_training STARTED
[03.04|14:17:48] JOB custom_initial_training RUNNING
[03.04|14:20:57] JOB custom_initial_training FINISHED
[03.04|14:20:57] JOB custom_initial_training SUCCESSFUL
<scm.params.plams.paramsjob.ParAMSResults at 0x7fae1d5130a0>
Set up the active learning job¶
Here the key new setting is the
ReasonableSimulationCriteria.GradientsUncertainty. This setting will
cause the MD simulation to instantly stop if the uncertainty is greater
than 1.0 eV/angstrom.
This is useful since the ML model is unlikely to give good predictions for the new types of structures encountered during the reactive MD.
In the summary log, such an event will be marked as “FAILED” with the reason “GRADIENTS_UNCERTAINTY”.
In order to use ML uncertainties, you need to train a committee model with at least 2 members. Here we set the commmittee size to 2. We also choose to train the 2 committee members in parallel. By default, they would be trained in sequence.
It is a good idea to train them in parallel if you have the computational resources to do so (for example, enough GPU memory).
When using uncertainty-based critiera, you may consider increasing the MaxAttemptsPerStep. Here, we stick with the default value of 15.
d_al = drivers.SimpleActiveLearning()
d_al.ActiveLearning.InitialReferenceData.Load.FromPreviousModel = True
d_al.ActiveLearning.Steps.Type = "Geometric"
d_al.ActiveLearning.Steps.Geometric.NumSteps = 5
d_al.ActiveLearning.Steps.Geometric.Start = 10
d_al.ActiveLearning.ReasonableSimulationCriteria.GradientsUncertainty.Enabled = True
d_al.ActiveLearning.ReasonableSimulationCriteria.GradientsUncertainty.MaxValue = 1.0 # eV/ang
d_al.ActiveLearning.SuccessCriteria.Forces.MinR2 = 0.4
d_al.ActiveLearning.MaxReferenceCalculationsPerAttempt = 3
d_al.ActiveLearning.MaxAttemptsPerStep = 15
d_al.MachineLearning.Backend = "M3GNet"
d_al.MachineLearning.LoadModel = os.path.abspath(paramsjob.results.path)
d_al.MachineLearning.CommitteeSize = 2
d_al.MachineLearning.MaxEpochs = 120
d_al.MachineLearning.M3GNet.LearningRate = 5e-4
d_al.MachineLearning.RunAMSAtEnd = False
d_al.ParallelLevels.CommitteeMembers = 2
set_reaction_boost(d_al)
d_al.Engine = e_dft
sal_job = SimpleActiveLearningJob(name="sal", driver=d_al, molecule=molecule_dict)
print(sal_job.get_input())
ActiveLearning
InitialReferenceData
Load
FromPreviousModel True
End
End
MaxReferenceCalculationsPerAttempt 3
ReasonableSimulationCriteria
GradientsUncertainty
Enabled True
MaxValue 1.0
End
End
Steps
Geometric
NumSteps 5
Start 10
End
Type Geometric
End
SuccessCriteria
Forces
MinR2 0.4
End
End
End
MachineLearning
Backend M3GNet
CommitteeSize 2
LoadModel /path/plams_workdir.002/custom_initial_training/results
M3GNet
LearningRate 0.0005
End
MaxEpochs 120
RunAMSAtEnd False
End
MolecularDynamics
InitialVelocities
Temperature 100.0
End
NSteps 3000
ReactionBoost
BondBreakingRestraints
Erf
MaxForce 0.05
End
Type Erf
End
BondMakingRestraints
Erf
MaxForce 0.12
End
Type Erf
End
Change LogForce
InitialFraction 0.05
NSteps 3000
TargetSystem final
Type Pair
End
Thermostat
Tau 5.0
Temperature 100.0
Type Berendsen
End
TimeStep 0.25
Trajectory
SamplingFreq 10
WriteBonds False
WriteMolecules False
End
End
ParallelLevels
CommitteeMembers 2
End
Task MolecularDynamics
Engine ADF
Basis
Type TZP
End
Occupations ElectronicTemperature=300 OptimizeSpinRound=0.05
Unrestricted True
XC
Dispersion GRIMME3 BJDAMP
GGA PBE
End
EndEngine
System
Atoms
C 1.7706024455 0.5099261992 0.1954040760
C 0.5407997618 0.1078450724 0.6002259183
C -0.6991001881 0.2752101814 -0.2193204192
H 2.6449981186 -0.1376064642 0.2891692003
H 1.8911990573 1.2634654944 -0.5900948315
H 0.4039536189 -0.4140897577 1.5514409421
H -1.5606977542 -0.1768812044 0.2902837692
H -0.6065440909 -0.1979378740 -1.2102651915
H -0.9295337746 1.3376911413 -0.3799046164
O 3.1225840267 2.0103760657 1.4948229694
H 2.9751374982 2.7751306018 2.0770419415
H 2.2389949314 1.5409219989 1.4471773663
End
End
System final
Atoms
C 2.1686443463 0.5464941815 0.4723864488
C 0.7390971298 -0.0070293802 0.7782564523
C -0.3560655234 0.7381224479 0.0595054883
H 2.8907079625 -0.1811906005 0.8761881668
H 2.3157834714 0.5522714680 -0.6266207675
H 0.7981691258 -1.0529976725 0.4091630194
H -1.2958639045 0.4296357496 0.5368432163
H -0.4039136209 0.4615873466 -1.0036755097
H -0.2802922202 1.8318359331 0.1421530387
O 2.4188223345 1.8223002981 1.0077125368
H 2.5409431029 2.4889805122 0.3092516933
H 0.4762175222 -0.1339653764 1.8246483976
End
End
sal_job.run(watch=True)
[03.04|14:22:56] JOB sal STARTED
[03.04|14:22:56] JOB sal RUNNING
[03.04|14:22:58] Simple Active Learning 2024.101, Nodes: 1, Procs: 8
[03.04|14:23:01] Composition of main system: C3H8O
[03.04|14:23:01] All REFERENCE calculations will be performed with the following ADF engine:
[03.04|14:23:01]
Engine adf
basis
type TZP
End
occupations ElectronicTemperature=300 OptimizeSpinRound=0.05
unrestricted True
xc
dispersion GRIMME3 BJDAMP
gga PBE
End
EndEngine
[03.04|14:23:01] The following are the settings for the to-be-trained MACHINE LEARNING model:
[03.04|14:23:01]
MachineLearning
Backend M3GNet
CommitteeSize 2
LoadModel /path/plams_workdir.002/custom_initial_training/results
M3GNet
LearningRate 0.0005
End
MaxEpochs 120
RunAMSAtEnd False
End
ParallelLevels
CommitteeMembers 2
End
[03.04|14:23:01] A committee model with 2 members will be trained.
[03.04|14:23:01] The ACTIVE LEARNING loop will contain 5 steps, using the following schema:
[03.04|14:23:01] Active Learning Step 1: 10 MD Steps (cumulative: 10)
[03.04|14:23:01] Active Learning Step 2: 31 MD Steps (cumulative: 41)
[03.04|14:23:01] Active Learning Step 3: 132 MD Steps (cumulative: 173)
[03.04|14:23:01] Active Learning Step 4: 547 MD Steps (cumulative: 720)
[03.04|14:23:01] Active Learning Step 5: 2280 MD Steps (cumulative: 3000)
[03.04|14:23:01] Total number of MD Steps: 3000
[03.04|14:23:01] Max attempts per active learning step: 15
[03.04|14:23:01]
[03.04|14:23:01] The directory /path/plams_workdir.002/custom_initial_training/results is of type RestartDirectoryType.PARAMS_RESULTS
[03.04|14:23:01] Successfully loaded previous ParAMS Job: from /path/plams_workdir.002/custom_initial_training/results
[03.04|14:23:01] Starting active learning loop...
[03.04|14:23:01] ##########################
[03.04|14:23:01] ### Step 1 / Attempt 1 ###
[03.04|14:23:01] ##########################
[03.04|14:23:01] MD Steps: 10 (cumulative: 10)
[03.04|14:23:01] Current engine settings:
[03.04|14:23:01]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/loaded_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/loaded_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:23:01] Running step1_attempt1_simulation...
[03.04|14:23:22] Job step1_attempt1_simulation finished
[03.04|14:23:22] Deleting files that are no longer needed...
[03.04|14:23:22] Energy uncertainty for final frame of step1_attempt1_simulation: 0.0778 eV
[03.04|14:23:22] 0.0065 eV/atom
[03.04|14:23:22] Forces uncertainty for final frame of step1_attempt1_simulation: 0.4190 eV/angstrom
[03.04|14:23:23] Launching reference calculation
[03.04|14:23:30] Reference calculation finished!
[03.04|14:23:30] Checking success for step1_attempt1
[03.04|14:23:30] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:23:30] CheckEnergy: normalization coefficient = 12
[03.04|14:23:30] CheckEnergy: Actual Threshold
[03.04|14:23:30] CheckEnergy: dE/12 -0.0182 0.2000 OK!
[03.04|14:23:30]
[03.04|14:23:30] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:23:30] CheckForces: ------------
[03.04|14:23:30] CheckForces: Reference job from step1_attempt1_reference_calc1
[03.04|14:23:30] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt1_simulation
[03.04|14:23:30] CheckForces: ------------
[03.04|14:23:30] CheckForces: Histogram of forces
[03.04|14:23:30] CheckForces: eV/Ang Ref Pred
[03.04|14:23:30] CheckForces: -3 0 0
[03.04|14:23:30] CheckForces: -2 2 0
[03.04|14:23:30] CheckForces: -1 13 18
[03.04|14:23:30] CheckForces: 0 19 18
[03.04|14:23:30] CheckForces: 1 1 0
[03.04|14:23:30] CheckForces: 2 1 0
[03.04|14:23:30] CheckForces: 3 0 0
[03.04|14:23:30] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:23:30] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:23:30] CheckForces: Ref Pred Delta Threshold
[03.04|14:23:30] CheckForces: 2.92 0.21 2.70 0.76
[03.04|14:23:30] CheckForces: -0.77 0.19 0.95 0.54
[03.04|14:23:30] CheckForces: -1.99 -0.23 1.76 0.64
[03.04|14:23:30] CheckForces: 0.74 0.03 0.71 0.54
[03.04|14:23:30] CheckForces: -1.39 -0.02 1.37 0.58
[03.04|14:23:30] CheckForces: 0.96 0.12 0.83 0.55
[03.04|14:23:30] CheckForces: ... and 6 more.
[03.04|14:23:30] CheckForces: Maximum deviation: 2.704 eV/angstrom
[03.04|14:23:30] CheckForces: Actual Threshold
[03.04|14:23:30] CheckForces: # > thr. 12 0 Not OK!
[03.04|14:23:30] CheckForces: MAE 0.473 0.30 Not OK!
[03.04|14:23:30] CheckForces: R^2 0.252 0.40 Not OK!
[03.04|14:23:30] CheckForces: --------------------
[03.04|14:23:30]
[03.04|14:23:30] Adding results from step1_attempt1_reference_calc1 to training set
[03.04|14:23:30] Current # training set entries: 23
[03.04|14:23:30] Current # validation set entries: 2
[03.04|14:23:30] Storing data in step1_attempt1_reference_data
[03.04|14:23:30] Deleting initial_reference_data
[03.04|14:23:30] Deleting step1_attempt1_reference_calc1
[03.04|14:23:30]
[03.04|14:23:30] Current (cumulative) timings:
[03.04|14:23:30] Time (s) Fraction
[03.04|14:23:30] Ref. calcs 7.54 0.261
[03.04|14:23:30] ML training 0.00 0.000
[03.04|14:23:30] Simulations 21.36 0.739
[03.04|14:23:30]
[03.04|14:23:30]
[03.04|14:23:31]
[03.04|14:23:31] --- Begin summary ---
[03.04|14:23:31] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:23:31] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:23:31] --- End summary ---
[03.04|14:23:31]
[03.04|14:23:31] Running more reference calculations....
[03.04|14:23:31] Running reference calculations on frames [6] from step1_attempt1_simulation/ams.rkf
[03.04|14:23:31] Calculating 1 frames in total
[03.04|14:23:31] Running step1_attempt1_reference_calc2
[03.04|14:23:38] Reference calculations finished!
[03.04|14:23:38] Adding results from step1_attempt1_reference_calc2 to validation set
[03.04|14:23:38] Current # training set entries: 23
[03.04|14:23:38] Current # validation set entries: 3
[03.04|14:23:38] Storing data in step1_attempt1_reference_data
[03.04|14:23:38] Deleting step1_attempt1_reference_calc2
[03.04|14:23:38] Launching reparametrization job: step1_attempt1_training
[03.04|14:23:43] JOB optimizer_001 STARTED
[03.04|14:23:43] Starting optimizer_001.prerun()
[03.04|14:23:43] JOB optimizer_002 STARTED
[03.04|14:23:43] optimizer_001.prerun() finished
[03.04|14:23:43] Starting optimizer_002.prerun()
[03.04|14:23:43] optimizer_002.prerun() finished
[03.04|14:23:43] JOB optimizer_001 RUNNING
[03.04|14:23:43] Executing optimizer_001.run
[03.04|14:23:43] JOB optimizer_002 RUNNING
[03.04|14:23:43] Executing optimizer_002.run
[03.04|14:23:43] Waiting for job optimizer_001 to finish
[03.04|14:24:49] training_set Optimizer: 002 Epoch: 0 Loss: 0.006936
[03.04|14:24:49] validation_set Optimizer: 002 Epoch: 0 Loss: 0.030481
[03.04|14:24:50] training_set Optimizer: 001 Epoch: 0 Loss: 0.006937
[03.04|14:24:50] validation_set Optimizer: 001 Epoch: 0 Loss: 0.037794
[03.04|14:24:52] training_set Optimizer: 002 Epoch: 10 Loss: 0.006427
[03.04|14:24:52] validation_set Optimizer: 002 Epoch: 10 Loss: 0.031368
[03.04|14:24:53] training_set Optimizer: 001 Epoch: 10 Loss: 0.006520
[03.04|14:24:53] validation_set Optimizer: 001 Epoch: 10 Loss: 0.026679
[03.04|14:24:55] training_set Optimizer: 002 Epoch: 20 Loss: 0.006483
[03.04|14:24:55] validation_set Optimizer: 002 Epoch: 20 Loss: 0.029778
[03.04|14:24:56] training_set Optimizer: 001 Epoch: 20 Loss: 0.006122
[03.04|14:24:56] validation_set Optimizer: 001 Epoch: 20 Loss: 0.024776
[03.04|14:24:57] training_set Optimizer: 002 Epoch: 30 Loss: 0.006156
[03.04|14:24:57] validation_set Optimizer: 002 Epoch: 30 Loss: 0.031093
[03.04|14:24:59] training_set Optimizer: 001 Epoch: 30 Loss: 0.006147
[03.04|14:24:59] validation_set Optimizer: 001 Epoch: 30 Loss: 0.022180
[03.04|14:25:00] training_set Optimizer: 002 Epoch: 40 Loss: 0.006422
[03.04|14:25:00] validation_set Optimizer: 002 Epoch: 40 Loss: 0.027520
[03.04|14:25:01] training_set Optimizer: 001 Epoch: 40 Loss: 0.006015
[03.04|14:25:01] validation_set Optimizer: 001 Epoch: 40 Loss: 0.021861
[03.04|14:25:03] training_set Optimizer: 002 Epoch: 50 Loss: 0.005607
[03.04|14:25:03] validation_set Optimizer: 002 Epoch: 50 Loss: 0.026729
[03.04|14:25:04] training_set Optimizer: 001 Epoch: 50 Loss: 0.006578
[03.04|14:25:04] validation_set Optimizer: 001 Epoch: 50 Loss: 0.021173
[03.04|14:25:06] training_set Optimizer: 002 Epoch: 60 Loss: 0.006000
[03.04|14:25:06] validation_set Optimizer: 002 Epoch: 60 Loss: 0.025323
[03.04|14:25:07] training_set Optimizer: 001 Epoch: 60 Loss: 0.006647
[03.04|14:25:07] validation_set Optimizer: 001 Epoch: 60 Loss: 0.026872
[03.04|14:25:08] training_set Optimizer: 002 Epoch: 70 Loss: 0.006276
[03.04|14:25:08] validation_set Optimizer: 002 Epoch: 70 Loss: 0.024150
[03.04|14:25:10] training_set Optimizer: 001 Epoch: 70 Loss: 0.006651
[03.04|14:25:10] validation_set Optimizer: 001 Epoch: 70 Loss: 0.022454
[03.04|14:25:11] training_set Optimizer: 002 Epoch: 80 Loss: 0.005477
[03.04|14:25:11] validation_set Optimizer: 002 Epoch: 80 Loss: 0.024143
[03.04|14:25:13] training_set Optimizer: 001 Epoch: 80 Loss: 0.006141
[03.04|14:25:13] validation_set Optimizer: 001 Epoch: 80 Loss: 0.023696
[03.04|14:25:14] training_set Optimizer: 002 Epoch: 90 Loss: 0.005072
[03.04|14:25:14] validation_set Optimizer: 002 Epoch: 90 Loss: 0.023812
[03.04|14:25:15] training_set Optimizer: 001 Epoch: 90 Loss: 0.005353
[03.04|14:25:15] validation_set Optimizer: 001 Epoch: 90 Loss: 0.024237
[03.04|14:25:17] training_set Optimizer: 002 Epoch: 100 Loss: 0.005131
[03.04|14:25:17] validation_set Optimizer: 002 Epoch: 100 Loss: 0.029612
[03.04|14:25:18] training_set Optimizer: 001 Epoch: 100 Loss: 0.006062
[03.04|14:25:18] validation_set Optimizer: 001 Epoch: 100 Loss: 0.022341
[03.04|14:25:20] training_set Optimizer: 002 Epoch: 110 Loss: 0.004923
[03.04|14:25:20] validation_set Optimizer: 002 Epoch: 110 Loss: 0.028610
[03.04|14:25:21] training_set Optimizer: 001 Epoch: 110 Loss: 0.005234
[03.04|14:25:21] validation_set Optimizer: 001 Epoch: 110 Loss: 0.022673
[03.04|14:25:24] Execution of optimizer_002.run finished with returncode 0
[03.04|14:25:24] JOB optimizer_002 FINISHED
[03.04|14:25:24] Starting optimizer_002.postrun()
[03.04|14:25:24] optimizer_002.postrun() finished
[03.04|14:25:24] JOB optimizer_002 SUCCESSFUL
[03.04|14:25:25] Execution of optimizer_001.run finished with returncode 0
[03.04|14:25:25] JOB optimizer_001 FINISHED
[03.04|14:25:25] Starting optimizer_001.postrun()
[03.04|14:25:25] optimizer_001.postrun() finished
[03.04|14:25:25] JOB optimizer_001 SUCCESSFUL
[03.04|14:25:25] PLAMS environment cleaned up successfully
[03.04|14:25:25] PLAMS run finished. Goodbye
[03.04|14:25:26] ParAMSResults
[03.04|14:25:26] Newly created parameter file/dir: step1_attempt1_training/results/optimization/optimizer_001/m3gnet
[03.04|14:25:26] Newly created parameter file/dir: step1_attempt1_training/results/optimization/optimizer_002/m3gnet
[03.04|14:25:26] Done!
[03.04|14:25:26] Deleting loaded_training
[03.04|14:25:26] ##########################
[03.04|14:25:26] ### Step 1 / Attempt 2 ###
[03.04|14:25:26] ##########################
[03.04|14:25:26] MD Steps: 10 (cumulative: 10)
[03.04|14:25:26] Current engine settings:
[03.04|14:25:26]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt1_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt1_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:25:26] Running step1_attempt2_simulation...
[03.04|14:25:46] Job step1_attempt2_simulation finished
[03.04|14:25:46] Deleting files that are no longer needed...
[03.04|14:25:46] Energy uncertainty for final frame of step1_attempt2_simulation: 0.0592 eV
[03.04|14:25:46] 0.0049 eV/atom
[03.04|14:25:46] Forces uncertainty for final frame of step1_attempt2_simulation: 0.3146 eV/angstrom
[03.04|14:25:47] Launching reference calculation
[03.04|14:25:54] Reference calculation finished!
[03.04|14:25:54] Checking success for step1_attempt2
[03.04|14:26:13] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:26:13] CheckEnergy: normalization coefficient = 12
[03.04|14:26:13] CheckEnergy: Actual Threshold
[03.04|14:26:13] CheckEnergy: dE/12 -0.0113 0.2000 OK!
[03.04|14:26:13] CheckEnergy: ddE/12 -0.0055 0.0050 Not OK! (relative to step1_attempt1_simulation:MDStep10)
[03.04|14:26:13]
[03.04|14:26:13] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:26:13] CheckForces: ------------
[03.04|14:26:13] CheckForces: Reference job from step1_attempt2_reference_calc1
[03.04|14:26:13] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt2_simulation
[03.04|14:26:13] CheckForces: ------------
[03.04|14:26:13] CheckForces: Histogram of forces
[03.04|14:26:13] CheckForces: eV/Ang Ref Pred
[03.04|14:26:13] CheckForces: -3 1 0
[03.04|14:26:13] CheckForces: -2 3 0
[03.04|14:26:13] CheckForces: -1 15 18
[03.04|14:26:13] CheckForces: 0 12 17
[03.04|14:26:13] CheckForces: 1 4 1
[03.04|14:26:13] CheckForces: 2 1 0
[03.04|14:26:13] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:26:13] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:26:13] CheckForces: Ref Pred Delta Threshold
[03.04|14:26:13] CheckForces: 0.21 0.88 0.67 0.51
[03.04|14:26:13] CheckForces: 2.49 1.21 1.27 0.70
[03.04|14:26:13] CheckForces: 1.09 0.18 0.90 0.56
[03.04|14:26:13] CheckForces: 0.83 0.12 0.71 0.54
[03.04|14:26:13] CheckForces: -1.24 -0.14 1.10 0.57
[03.04|14:26:13] CheckForces: 1.01 0.11 0.89 0.55
[03.04|14:26:13] CheckForces: ... and 9 more.
[03.04|14:26:13] CheckForces: Maximum deviation: 1.476 eV/angstrom
[03.04|14:26:13] CheckForces: Actual Threshold
[03.04|14:26:13] CheckForces: # > thr. 15 0 Not OK!
[03.04|14:26:13] CheckForces: MAE 0.495 0.30 Not OK!
[03.04|14:26:13] CheckForces: R^2 0.503 0.40 OK!
[03.04|14:26:13] CheckForces: --------------------
[03.04|14:26:13]
[03.04|14:26:13] Adding results from step1_attempt2_reference_calc1 to training set
[03.04|14:26:13] Current # training set entries: 24
[03.04|14:26:13] Current # validation set entries: 3
[03.04|14:26:13] Storing data in step1_attempt2_reference_data
[03.04|14:26:13] Deleting step1_attempt1_reference_data
[03.04|14:26:13] Deleting step1_attempt2_reference_calc1
[03.04|14:26:13]
[03.04|14:26:13] Current (cumulative) timings:
[03.04|14:26:13] Time (s) Fraction
[03.04|14:26:13] Ref. calcs 22.42 0.131
[03.04|14:26:13] ML training 107.62 0.627
[03.04|14:26:13] Simulations 41.66 0.243
[03.04|14:26:13]
[03.04|14:26:13]
[03.04|14:26:13]
[03.04|14:26:13] --- Begin summary ---
[03.04|14:26:13] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:26:13] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:26:13] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:26:13] --- End summary ---
[03.04|14:26:13]
[03.04|14:26:13] Running more reference calculations....
[03.04|14:26:14] Running reference calculations on frames [6] from step1_attempt2_simulation/ams.rkf
[03.04|14:26:14] Calculating 1 frames in total
[03.04|14:26:14] Running step1_attempt2_reference_calc2
[03.04|14:26:20] Reference calculations finished!
[03.04|14:26:20] Adding results from step1_attempt2_reference_calc2 to validation set
[03.04|14:26:20] Current # training set entries: 24
[03.04|14:26:20] Current # validation set entries: 4
[03.04|14:26:20] Storing data in step1_attempt2_reference_data
[03.04|14:26:21] Deleting step1_attempt2_reference_calc2
[03.04|14:26:21] Launching reparametrization job: step1_attempt2_training
[03.04|14:26:25] JOB optimizer_001 STARTED
[03.04|14:26:25] JOB optimizer_002 STARTED
[03.04|14:26:25] Starting optimizer_001.prerun()
[03.04|14:26:25] optimizer_001.prerun() finished
[03.04|14:26:25] Starting optimizer_002.prerun()
[03.04|14:26:25] optimizer_002.prerun() finished
[03.04|14:26:25] JOB optimizer_002 RUNNING
[03.04|14:26:25] Executing optimizer_002.run
[03.04|14:26:25] JOB optimizer_001 RUNNING
[03.04|14:26:25] Executing optimizer_001.run
[03.04|14:26:25] Waiting for job optimizer_001 to finish
[03.04|14:27:30] training_set Optimizer: 001 Epoch: 0 Loss: 0.007304
[03.04|14:27:30] validation_set Optimizer: 001 Epoch: 0 Loss: 0.046984
[03.04|14:27:32] training_set Optimizer: 002 Epoch: 0 Loss: 0.006948
[03.04|14:27:32] validation_set Optimizer: 002 Epoch: 0 Loss: 0.051815
[03.04|14:27:32] training_set Optimizer: 001 Epoch: 10 Loss: 0.006385
[03.04|14:27:32] validation_set Optimizer: 001 Epoch: 10 Loss: 0.025930
[03.04|14:27:34] training_set Optimizer: 002 Epoch: 10 Loss: 0.005766
[03.04|14:27:34] validation_set Optimizer: 002 Epoch: 10 Loss: 0.029463
[03.04|14:27:35] training_set Optimizer: 001 Epoch: 20 Loss: 0.006034
[03.04|14:27:35] validation_set Optimizer: 001 Epoch: 20 Loss: 0.023871
[03.04|14:27:37] training_set Optimizer: 002 Epoch: 20 Loss: 0.005507
[03.04|14:27:37] validation_set Optimizer: 002 Epoch: 20 Loss: 0.032861
[03.04|14:27:38] training_set Optimizer: 001 Epoch: 30 Loss: 0.006284
[03.04|14:27:38] validation_set Optimizer: 001 Epoch: 30 Loss: 0.023070
[03.04|14:27:40] training_set Optimizer: 002 Epoch: 30 Loss: 0.005457
[03.04|14:27:40] validation_set Optimizer: 002 Epoch: 30 Loss: 0.032226
[03.04|14:27:41] training_set Optimizer: 001 Epoch: 40 Loss: 0.005575
[03.04|14:27:41] validation_set Optimizer: 001 Epoch: 40 Loss: 0.023798
[03.04|14:27:43] training_set Optimizer: 002 Epoch: 40 Loss: 0.005106
[03.04|14:27:43] validation_set Optimizer: 002 Epoch: 40 Loss: 0.028401
[03.04|14:27:43] training_set Optimizer: 001 Epoch: 50 Loss: 0.005270
[03.04|14:27:43] validation_set Optimizer: 001 Epoch: 50 Loss: 0.024840
[03.04|14:27:45] training_set Optimizer: 002 Epoch: 50 Loss: 0.004986
[03.04|14:27:45] validation_set Optimizer: 002 Epoch: 50 Loss: 0.027495
[03.04|14:27:46] training_set Optimizer: 001 Epoch: 60 Loss: 0.005224
[03.04|14:27:46] validation_set Optimizer: 001 Epoch: 60 Loss: 0.024221
[03.04|14:27:48] training_set Optimizer: 002 Epoch: 60 Loss: 0.004681
[03.04|14:27:48] validation_set Optimizer: 002 Epoch: 60 Loss: 0.026728
[03.04|14:27:49] training_set Optimizer: 001 Epoch: 70 Loss: 0.004880
[03.04|14:27:49] validation_set Optimizer: 001 Epoch: 70 Loss: 0.025811
[03.04|14:27:51] training_set Optimizer: 002 Epoch: 70 Loss: 0.004540
[03.04|14:27:51] validation_set Optimizer: 002 Epoch: 70 Loss: 0.026011
[03.04|14:27:52] training_set Optimizer: 001 Epoch: 80 Loss: 0.004664
[03.04|14:27:52] validation_set Optimizer: 001 Epoch: 80 Loss: 0.025285
[03.04|14:27:54] training_set Optimizer: 002 Epoch: 80 Loss: 0.004339
[03.04|14:27:54] validation_set Optimizer: 002 Epoch: 80 Loss: 0.025223
[03.04|14:27:55] training_set Optimizer: 001 Epoch: 90 Loss: 0.004589
[03.04|14:27:55] validation_set Optimizer: 001 Epoch: 90 Loss: 0.024437
[03.04|14:27:57] training_set Optimizer: 002 Epoch: 90 Loss: 0.004106
[03.04|14:27:57] validation_set Optimizer: 002 Epoch: 90 Loss: 0.019066
[03.04|14:27:57] training_set Optimizer: 001 Epoch: 100 Loss: 0.004266
[03.04|14:27:57] validation_set Optimizer: 001 Epoch: 100 Loss: 0.026022
[03.04|14:27:59] training_set Optimizer: 002 Epoch: 100 Loss: 0.004005
[03.04|14:27:59] validation_set Optimizer: 002 Epoch: 100 Loss: 0.029166
[03.04|14:28:00] training_set Optimizer: 001 Epoch: 110 Loss: 0.004438
[03.04|14:28:00] validation_set Optimizer: 001 Epoch: 110 Loss: 0.024953
[03.04|14:28:02] training_set Optimizer: 002 Epoch: 110 Loss: 0.004146
[03.04|14:28:02] validation_set Optimizer: 002 Epoch: 110 Loss: 0.018678
[03.04|14:28:04] Execution of optimizer_001.run finished with returncode 0
[03.04|14:28:05] JOB optimizer_001 FINISHED
[03.04|14:28:05] Starting optimizer_001.postrun()
[03.04|14:28:05] optimizer_001.postrun() finished
[03.04|14:28:05] JOB optimizer_001 SUCCESSFUL
[03.04|14:28:05] Waiting for job optimizer_002 to finish
[03.04|14:28:06] Execution of optimizer_002.run finished with returncode 0
[03.04|14:28:06] JOB optimizer_002 FINISHED
[03.04|14:28:06] Starting optimizer_002.postrun()
[03.04|14:28:06] optimizer_002.postrun() finished
[03.04|14:28:06] JOB optimizer_002 SUCCESSFUL
[03.04|14:28:06] PLAMS environment cleaned up successfully
[03.04|14:28:06] PLAMS run finished. Goodbye
[03.04|14:28:07] ParAMSResults
[03.04|14:28:07] Newly created parameter file/dir: step1_attempt2_training/results/optimization/optimizer_001/m3gnet
[03.04|14:28:07] Newly created parameter file/dir: step1_attempt2_training/results/optimization/optimizer_002/m3gnet
[03.04|14:28:07] Done!
[03.04|14:28:07] Deleting step1_attempt1_training
[03.04|14:28:07] ##########################
[03.04|14:28:07] ### Step 1 / Attempt 3 ###
[03.04|14:28:07] ##########################
[03.04|14:28:07] MD Steps: 10 (cumulative: 10)
[03.04|14:28:07] Current engine settings:
[03.04|14:28:07]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:28:07] Running step1_attempt3_simulation...
[03.04|14:28:33] Job step1_attempt3_simulation finished
[03.04|14:28:33] Deleting files that are no longer needed...
[03.04|14:28:33] Energy uncertainty for final frame of step1_attempt3_simulation: 0.2220 eV
[03.04|14:28:33] 0.0185 eV/atom
[03.04|14:28:33] Forces uncertainty for final frame of step1_attempt3_simulation: 0.3262 eV/angstrom
[03.04|14:28:33] Launching reference calculation
[03.04|14:28:40] Reference calculation finished!
[03.04|14:28:40] Checking success for step1_attempt3
[03.04|14:29:00] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:29:00] CheckEnergy: normalization coefficient = 12
[03.04|14:29:00] CheckEnergy: Actual Threshold
[03.04|14:29:00] CheckEnergy: dE/12 0.0017 0.2000 OK!
[03.04|14:29:00] CheckEnergy: ddE/12 0.0016 0.0050 OK! (relative to step1_attempt2_simulation:MDStep10)
[03.04|14:29:00]
[03.04|14:29:00] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:29:00] CheckForces: ------------
[03.04|14:29:00] CheckForces: Reference job from step1_attempt3_reference_calc1
[03.04|14:29:00] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt3_simulation
[03.04|14:29:00] CheckForces: ------------
[03.04|14:29:00] CheckForces: Histogram of forces
[03.04|14:29:00] CheckForces: eV/Ang Ref Pred
[03.04|14:29:00] CheckForces: -2 3 0
[03.04|14:29:00] CheckForces: -1 17 24
[03.04|14:29:00] CheckForces: 0 14 10
[03.04|14:29:00] CheckForces: 1 2 2
[03.04|14:29:00] CheckForces: 2 0 0
[03.04|14:29:00] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:29:00] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:29:00] CheckForces: Ref Pred Delta Threshold
[03.04|14:29:00] CheckForces: -0.71 -0.15 0.56 0.54
[03.04|14:29:00] CheckForces: 0.69 -0.05 0.74 0.53
[03.04|14:29:00] CheckForces: -1.11 -0.32 0.79 0.56
[03.04|14:29:00] CheckForces: -0.35 0.17 0.52 0.52
[03.04|14:29:00] CheckForces: 0.36 -0.36 0.72 0.52
[03.04|14:29:00] CheckForces: -1.06 -0.23 0.83 0.56
[03.04|14:29:00] CheckForces: -0.72 -0.13 0.59 0.54
[03.04|14:29:00] CheckForces: Maximum deviation: 0.834 eV/angstrom
[03.04|14:29:00] CheckForces: Actual Threshold
[03.04|14:29:00] CheckForces: # > thr. 7 0 Not OK!
[03.04|14:29:00] CheckForces: MAE 0.285 0.30 OK!
[03.04|14:29:00] CheckForces: R^2 0.679 0.40 OK!
[03.04|14:29:00] CheckForces: --------------------
[03.04|14:29:00]
[03.04|14:29:00] Adding results from step1_attempt3_reference_calc1 to training set
[03.04|14:29:00] Current # training set entries: 25
[03.04|14:29:00] Current # validation set entries: 4
[03.04|14:29:00] Storing data in step1_attempt3_reference_data
[03.04|14:29:00] Deleting step1_attempt2_reference_data
[03.04|14:29:00] Deleting step1_attempt3_reference_calc1
[03.04|14:29:00]
[03.04|14:29:00] Current (cumulative) timings:
[03.04|14:29:00] Time (s) Fraction
[03.04|14:29:00] Ref. calcs 36.40 0.114
[03.04|14:29:00] ML training 214.01 0.673
[03.04|14:29:00] Simulations 67.63 0.213
[03.04|14:29:00]
[03.04|14:29:00]
[03.04|14:29:00]
[03.04|14:29:00] --- Begin summary ---
[03.04|14:29:00] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:29:00] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:29:00] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:29:00] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:29:00] --- End summary ---
[03.04|14:29:00]
[03.04|14:29:00] Running more reference calculations....
[03.04|14:29:00] Running reference calculations on frames [6] from step1_attempt3_simulation/ams.rkf
[03.04|14:29:00] Calculating 1 frames in total
[03.04|14:29:00] Running step1_attempt3_reference_calc2
[03.04|14:29:07] Reference calculations finished!
[03.04|14:29:07] Adding results from step1_attempt3_reference_calc2 to validation set
[03.04|14:29:07] Current # training set entries: 25
[03.04|14:29:07] Current # validation set entries: 5
[03.04|14:29:07] Storing data in step1_attempt3_reference_data
[03.04|14:29:08] Deleting step1_attempt3_reference_calc2
[03.04|14:29:08] Launching reparametrization job: step1_attempt3_training
[03.04|14:29:12] JOB optimizer_001 STARTED
[03.04|14:29:12] Starting optimizer_001.prerun()
[03.04|14:29:12] JOB optimizer_002 STARTED
[03.04|14:29:12] optimizer_001.prerun() finished
[03.04|14:29:12] Starting optimizer_002.prerun()
[03.04|14:29:12] optimizer_002.prerun() finished
[03.04|14:29:12] JOB optimizer_002 RUNNING
[03.04|14:29:12] Executing optimizer_002.run
[03.04|14:29:12] JOB optimizer_001 RUNNING
[03.04|14:29:12] Executing optimizer_001.run
[03.04|14:29:12] Waiting for job optimizer_001 to finish
[03.04|14:29:56] training_set Optimizer: 002 Epoch: 0 Loss: 0.005156
[03.04|14:29:56] validation_set Optimizer: 002 Epoch: 0 Loss: 0.038036
[03.04|14:29:57] training_set Optimizer: 001 Epoch: 0 Loss: 0.005570
[03.04|14:29:57] validation_set Optimizer: 001 Epoch: 0 Loss: 0.053884
[03.04|14:29:59] training_set Optimizer: 002 Epoch: 10 Loss: 0.004037
[03.04|14:29:59] validation_set Optimizer: 002 Epoch: 10 Loss: 0.020110
[03.04|14:30:00] training_set Optimizer: 001 Epoch: 10 Loss: 0.004254
[03.04|14:30:00] validation_set Optimizer: 001 Epoch: 10 Loss: 0.020539
[03.04|14:30:02] training_set Optimizer: 002 Epoch: 20 Loss: 0.004005
[03.04|14:30:02] validation_set Optimizer: 002 Epoch: 20 Loss: 0.020733
[03.04|14:30:03] training_set Optimizer: 001 Epoch: 20 Loss: 0.004050
[03.04|14:30:03] validation_set Optimizer: 001 Epoch: 20 Loss: 0.023235
[03.04|14:30:04] training_set Optimizer: 002 Epoch: 30 Loss: 0.004000
[03.04|14:30:04] validation_set Optimizer: 002 Epoch: 30 Loss: 0.029901
[03.04|14:30:05] training_set Optimizer: 001 Epoch: 30 Loss: 0.004067
[03.04|14:30:05] validation_set Optimizer: 001 Epoch: 30 Loss: 0.027147
[03.04|14:30:07] training_set Optimizer: 002 Epoch: 40 Loss: 0.003812
[03.04|14:30:07] validation_set Optimizer: 002 Epoch: 40 Loss: 0.021091
[03.04|14:30:08] training_set Optimizer: 001 Epoch: 40 Loss: 0.003853
[03.04|14:30:08] validation_set Optimizer: 001 Epoch: 40 Loss: 0.027924
[03.04|14:30:10] training_set Optimizer: 002 Epoch: 50 Loss: 0.003719
[03.04|14:30:10] validation_set Optimizer: 002 Epoch: 50 Loss: 0.025166
[03.04|14:30:11] training_set Optimizer: 001 Epoch: 50 Loss: 0.003815
[03.04|14:30:11] validation_set Optimizer: 001 Epoch: 50 Loss: 0.020120
[03.04|14:30:13] training_set Optimizer: 002 Epoch: 60 Loss: 0.003659
[03.04|14:30:13] validation_set Optimizer: 002 Epoch: 60 Loss: 0.021571
[03.04|14:30:14] training_set Optimizer: 001 Epoch: 60 Loss: 0.003819
[03.04|14:30:14] validation_set Optimizer: 001 Epoch: 60 Loss: 0.019822
[03.04|14:30:16] training_set Optimizer: 002 Epoch: 70 Loss: 0.003544
[03.04|14:30:16] validation_set Optimizer: 002 Epoch: 70 Loss: 0.023276
[03.04|14:30:17] training_set Optimizer: 001 Epoch: 70 Loss: 0.003647
[03.04|14:30:17] validation_set Optimizer: 001 Epoch: 70 Loss: 0.020353
[03.04|14:30:18] training_set Optimizer: 002 Epoch: 80 Loss: 0.003380
[03.04|14:30:18] validation_set Optimizer: 002 Epoch: 80 Loss: 0.016483
[03.04|14:30:20] training_set Optimizer: 001 Epoch: 80 Loss: 0.003583
[03.04|14:30:20] validation_set Optimizer: 001 Epoch: 80 Loss: 0.017632
[03.04|14:30:21] training_set Optimizer: 002 Epoch: 90 Loss: 0.003357
[03.04|14:30:21] validation_set Optimizer: 002 Epoch: 90 Loss: 0.025523
[03.04|14:30:22] training_set Optimizer: 001 Epoch: 90 Loss: 0.003511
[03.04|14:30:22] validation_set Optimizer: 001 Epoch: 90 Loss: 0.021776
[03.04|14:30:24] training_set Optimizer: 002 Epoch: 100 Loss: 0.003175
[03.04|14:30:24] validation_set Optimizer: 002 Epoch: 100 Loss: 0.014952
[03.04|14:30:25] training_set Optimizer: 001 Epoch: 100 Loss: 0.003454
[03.04|14:30:25] validation_set Optimizer: 001 Epoch: 100 Loss: 0.025210
[03.04|14:30:27] training_set Optimizer: 002 Epoch: 110 Loss: 0.003179
[03.04|14:30:27] validation_set Optimizer: 002 Epoch: 110 Loss: 0.019120
[03.04|14:30:28] training_set Optimizer: 001 Epoch: 110 Loss: 0.003479
[03.04|14:30:28] validation_set Optimizer: 001 Epoch: 110 Loss: 0.025523
[03.04|14:30:30] Execution of optimizer_002.run finished with returncode 0
[03.04|14:30:31] JOB optimizer_002 FINISHED
[03.04|14:30:31] Starting optimizer_002.postrun()
[03.04|14:30:31] optimizer_002.postrun() finished
[03.04|14:30:31] JOB optimizer_002 SUCCESSFUL
[03.04|14:30:31] Execution of optimizer_001.run finished with returncode 0
[03.04|14:30:32] JOB optimizer_001 FINISHED
[03.04|14:30:32] Starting optimizer_001.postrun()
[03.04|14:30:32] optimizer_001.postrun() finished
[03.04|14:30:32] JOB optimizer_001 SUCCESSFUL
[03.04|14:30:32] PLAMS environment cleaned up successfully
[03.04|14:30:32] PLAMS run finished. Goodbye
[03.04|14:30:32] ParAMSResults
[03.04|14:30:32] Newly created parameter file/dir: step1_attempt3_training/results/optimization/optimizer_001/m3gnet
[03.04|14:30:32] Newly created parameter file/dir: step1_attempt3_training/results/optimization/optimizer_002/m3gnet
[03.04|14:30:32] Done!
[03.04|14:30:32] Deleting step1_attempt2_training
[03.04|14:30:32] ##########################
[03.04|14:30:32] ### Step 1 / Attempt 4 ###
[03.04|14:30:32] ##########################
[03.04|14:30:32] MD Steps: 10 (cumulative: 10)
[03.04|14:30:32] Current engine settings:
[03.04|14:30:32]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt3_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt3_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:30:32] Running step1_attempt4_simulation...
[03.04|14:30:53] Job step1_attempt4_simulation finished
[03.04|14:30:53] Deleting files that are no longer needed...
[03.04|14:30:53] Energy uncertainty for final frame of step1_attempt4_simulation: 0.0401 eV
[03.04|14:30:53] 0.0033 eV/atom
[03.04|14:30:53] Forces uncertainty for final frame of step1_attempt4_simulation: 0.2426 eV/angstrom
[03.04|14:30:53] Launching reference calculation
[03.04|14:31:00] Reference calculation finished!
[03.04|14:31:00] Checking success for step1_attempt4
[03.04|14:31:20] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:31:20] CheckEnergy: normalization coefficient = 12
[03.04|14:31:20] CheckEnergy: Actual Threshold
[03.04|14:31:20] CheckEnergy: dE/12 -0.0114 0.2000 OK!
[03.04|14:31:20] CheckEnergy: ddE/12 -0.0035 0.0050 OK! (relative to step1_attempt3_simulation:MDStep10)
[03.04|14:31:20]
[03.04|14:31:20] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:31:20] CheckForces: ------------
[03.04|14:31:20] CheckForces: Reference job from step1_attempt4_reference_calc1
[03.04|14:31:20] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt4_simulation
[03.04|14:31:20] CheckForces: ------------
[03.04|14:31:20] CheckForces: Histogram of forces
[03.04|14:31:20] CheckForces: eV/Ang Ref Pred
[03.04|14:31:20] CheckForces: -2 4 1
[03.04|14:31:20] CheckForces: -1 14 17
[03.04|14:31:20] CheckForces: 0 16 16
[03.04|14:31:20] CheckForces: 1 2 2
[03.04|14:31:20] CheckForces: 2 0 0
[03.04|14:31:20] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:31:20] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:31:20] CheckForces: Ref Pred Delta Threshold
[03.04|14:31:20] CheckForces: -1.16 -0.45 0.71 0.57
[03.04|14:31:20] CheckForces: Maximum deviation: 0.709 eV/angstrom
[03.04|14:31:20] CheckForces: Actual Threshold
[03.04|14:31:20] CheckForces: # > thr. 1 0 Not OK!
[03.04|14:31:20] CheckForces: MAE 0.191 0.30 OK!
[03.04|14:31:20] CheckForces: R^2 0.861 0.40 OK!
[03.04|14:31:20] CheckForces: --------------------
[03.04|14:31:20]
[03.04|14:31:20] Adding results from step1_attempt4_reference_calc1 to training set
[03.04|14:31:20] Current # training set entries: 26
[03.04|14:31:20] Current # validation set entries: 5
[03.04|14:31:20] Storing data in step1_attempt4_reference_data
[03.04|14:31:20] Deleting step1_attempt3_reference_data
[03.04|14:31:20] Deleting step1_attempt4_reference_calc1
[03.04|14:31:20]
[03.04|14:31:20] Current (cumulative) timings:
[03.04|14:31:20] Time (s) Fraction
[03.04|14:31:20] Ref. calcs 51.32 0.117
[03.04|14:31:20] ML training 298.91 0.682
[03.04|14:31:20] Simulations 87.97 0.201
[03.04|14:31:20]
[03.04|14:31:20]
[03.04|14:31:20]
[03.04|14:31:20] --- Begin summary ---
[03.04|14:31:20] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:31:20] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:31:20] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:31:20] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:31:20] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:31:20] --- End summary ---
[03.04|14:31:20]
[03.04|14:31:20] Running more reference calculations....
[03.04|14:31:20] Running reference calculations on frames [6] from step1_attempt4_simulation/ams.rkf
[03.04|14:31:20] Calculating 1 frames in total
[03.04|14:31:20] Running step1_attempt4_reference_calc2
[03.04|14:31:28] Reference calculations finished!
[03.04|14:31:28] Adding results from step1_attempt4_reference_calc2 to validation set
[03.04|14:31:28] Current # training set entries: 26
[03.04|14:31:28] Current # validation set entries: 6
[03.04|14:31:28] Storing data in step1_attempt4_reference_data
[03.04|14:31:28] Deleting step1_attempt4_reference_calc2
[03.04|14:31:28] Launching reparametrization job: step1_attempt4_training
[03.04|14:31:32] JOB optimizer_001 STARTED
[03.04|14:31:32] Starting optimizer_001.prerun()
[03.04|14:31:32] JOB optimizer_002 STARTED
[03.04|14:31:32] optimizer_001.prerun() finished
[03.04|14:31:32] Starting optimizer_002.prerun()
[03.04|14:31:32] optimizer_002.prerun() finished
[03.04|14:31:32] JOB optimizer_001 RUNNING
[03.04|14:31:32] Executing optimizer_001.run
[03.04|14:31:32] JOB optimizer_002 RUNNING
[03.04|14:31:32] Executing optimizer_002.run
[03.04|14:31:32] Waiting for job optimizer_001 to finish
[03.04|14:32:41] training_set Optimizer: 002 Epoch: 0 Loss: 0.005511
[03.04|14:32:41] validation_set Optimizer: 002 Epoch: 0 Loss: 0.027907
[03.04|14:32:41] training_set Optimizer: 001 Epoch: 0 Loss: 0.005254
[03.04|14:32:41] validation_set Optimizer: 001 Epoch: 0 Loss: 0.026963
[03.04|14:32:44] training_set Optimizer: 002 Epoch: 10 Loss: 0.002896
[03.04|14:32:44] validation_set Optimizer: 002 Epoch: 10 Loss: 0.015149
[03.04|14:32:45] training_set Optimizer: 001 Epoch: 10 Loss: 0.003824
[03.04|14:32:45] validation_set Optimizer: 001 Epoch: 10 Loss: 0.023527
[03.04|14:32:47] training_set Optimizer: 002 Epoch: 20 Loss: 0.002879
[03.04|14:32:47] validation_set Optimizer: 002 Epoch: 20 Loss: 0.020222
[03.04|14:32:48] training_set Optimizer: 001 Epoch: 20 Loss: 0.003705
[03.04|14:32:48] validation_set Optimizer: 001 Epoch: 20 Loss: 0.020502
[03.04|14:32:51] training_set Optimizer: 002 Epoch: 30 Loss: 0.002951
[03.04|14:32:51] validation_set Optimizer: 002 Epoch: 30 Loss: 0.029683
[03.04|14:32:52] training_set Optimizer: 001 Epoch: 30 Loss: 0.003465
[03.04|14:32:52] validation_set Optimizer: 001 Epoch: 30 Loss: 0.044133
[03.04|14:32:54] training_set Optimizer: 002 Epoch: 40 Loss: 0.002900
[03.04|14:32:54] validation_set Optimizer: 002 Epoch: 40 Loss: 0.020275
[03.04|14:32:55] training_set Optimizer: 001 Epoch: 40 Loss: 0.003235
[03.04|14:32:55] validation_set Optimizer: 001 Epoch: 40 Loss: 0.022742
[03.04|14:32:58] training_set Optimizer: 002 Epoch: 50 Loss: 0.003480
[03.04|14:32:58] validation_set Optimizer: 002 Epoch: 50 Loss: 0.016454
[03.04|14:32:58] training_set Optimizer: 001 Epoch: 50 Loss: 0.003069
[03.04|14:32:58] validation_set Optimizer: 001 Epoch: 50 Loss: 0.029184
[03.04|14:33:01] training_set Optimizer: 002 Epoch: 60 Loss: 0.002845
[03.04|14:33:01] validation_set Optimizer: 002 Epoch: 60 Loss: 0.016220
[03.04|14:33:02] training_set Optimizer: 001 Epoch: 60 Loss: 0.003540
[03.04|14:33:02] validation_set Optimizer: 001 Epoch: 60 Loss: 0.018955
[03.04|14:33:05] training_set Optimizer: 002 Epoch: 70 Loss: 0.002912
[03.04|14:33:05] validation_set Optimizer: 002 Epoch: 70 Loss: 0.020131
[03.04|14:33:05] training_set Optimizer: 001 Epoch: 70 Loss: 0.003062
[03.04|14:33:05] validation_set Optimizer: 001 Epoch: 70 Loss: 0.022878
[03.04|14:33:08] training_set Optimizer: 002 Epoch: 80 Loss: 0.003109
[03.04|14:33:08] validation_set Optimizer: 002 Epoch: 80 Loss: 0.027322
[03.04|14:33:09] training_set Optimizer: 001 Epoch: 80 Loss: 0.004377
[03.04|14:33:09] validation_set Optimizer: 001 Epoch: 80 Loss: 0.017507
[03.04|14:33:11] training_set Optimizer: 002 Epoch: 90 Loss: 0.002947
[03.04|14:33:11] validation_set Optimizer: 002 Epoch: 90 Loss: 0.015082
[03.04|14:33:12] training_set Optimizer: 001 Epoch: 90 Loss: 0.003774
[03.04|14:33:12] validation_set Optimizer: 001 Epoch: 90 Loss: 0.018589
[03.04|14:33:15] training_set Optimizer: 002 Epoch: 100 Loss: 0.002837
[03.04|14:33:15] validation_set Optimizer: 002 Epoch: 100 Loss: 0.021636
[03.04|14:33:16] training_set Optimizer: 001 Epoch: 100 Loss: 0.002746
[03.04|14:33:16] validation_set Optimizer: 001 Epoch: 100 Loss: 0.016368
[03.04|14:33:18] training_set Optimizer: 002 Epoch: 110 Loss: 0.002755
[03.04|14:33:18] validation_set Optimizer: 002 Epoch: 110 Loss: 0.018310
[03.04|14:33:19] training_set Optimizer: 001 Epoch: 110 Loss: 0.003492
[03.04|14:33:19] validation_set Optimizer: 001 Epoch: 110 Loss: 0.018284
[03.04|14:33:23] Execution of optimizer_002.run finished with returncode 0
[03.04|14:33:23] JOB optimizer_002 FINISHED
[03.04|14:33:23] Starting optimizer_002.postrun()
[03.04|14:33:23] optimizer_002.postrun() finished
[03.04|14:33:23] JOB optimizer_002 SUCCESSFUL
[03.04|14:33:24] Execution of optimizer_001.run finished with returncode 0
[03.04|14:33:24] JOB optimizer_001 FINISHED
[03.04|14:33:24] Starting optimizer_001.postrun()
[03.04|14:33:24] optimizer_001.postrun() finished
[03.04|14:33:24] JOB optimizer_001 SUCCESSFUL
[03.04|14:33:24] PLAMS environment cleaned up successfully
[03.04|14:33:24] PLAMS run finished. Goodbye
[03.04|14:33:25] ParAMSResults
[03.04|14:33:25] Newly created parameter file/dir: step1_attempt4_training/results/optimization/optimizer_001/m3gnet
[03.04|14:33:25] Newly created parameter file/dir: step1_attempt4_training/results/optimization/optimizer_002/m3gnet
[03.04|14:33:25] Done!
[03.04|14:33:25] Deleting step1_attempt3_training
[03.04|14:33:25] ##########################
[03.04|14:33:25] ### Step 1 / Attempt 5 ###
[03.04|14:33:25] ##########################
[03.04|14:33:25] MD Steps: 10 (cumulative: 10)
[03.04|14:33:25] Current engine settings:
[03.04|14:33:25]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt4_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt4_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:33:25] Running step1_attempt5_simulation...
[03.04|14:33:44] Job step1_attempt5_simulation finished
[03.04|14:33:44] Deleting files that are no longer needed...
[03.04|14:33:44] Energy uncertainty for final frame of step1_attempt5_simulation: 0.0352 eV
[03.04|14:33:44] 0.0029 eV/atom
[03.04|14:33:44] Forces uncertainty for final frame of step1_attempt5_simulation: 0.2593 eV/angstrom
[03.04|14:33:45] Launching reference calculation
[03.04|14:33:52] Reference calculation finished!
[03.04|14:33:52] Checking success for step1_attempt5
[03.04|14:34:11] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:34:11] CheckEnergy: normalization coefficient = 12
[03.04|14:34:11] CheckEnergy: Actual Threshold
[03.04|14:34:11] CheckEnergy: dE/12 -0.0112 0.2000 OK!
[03.04|14:34:11] CheckEnergy: ddE/12 -0.0039 0.0050 OK! (relative to step1_attempt4_simulation:MDStep10)
[03.04|14:34:11]
[03.04|14:34:11] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:34:11] CheckForces: ------------
[03.04|14:34:11] CheckForces: Reference job from step1_attempt5_reference_calc1
[03.04|14:34:11] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt5_simulation
[03.04|14:34:11] CheckForces: ------------
[03.04|14:34:11] CheckForces: Histogram of forces
[03.04|14:34:11] CheckForces: eV/Ang Ref Pred
[03.04|14:34:11] CheckForces: -2 1 0
[03.04|14:34:11] CheckForces: -1 20 14
[03.04|14:34:11] CheckForces: 0 13 21
[03.04|14:34:11] CheckForces: 1 2 0
[03.04|14:34:11] CheckForces: 2 0 1
[03.04|14:34:11] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:34:11] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:34:11] CheckForces: Ref Pred Delta Threshold
[03.04|14:34:11] CheckForces: -0.39 0.21 0.59 0.52
[03.04|14:34:11] CheckForces: -1.09 -0.48 0.61 0.56
[03.04|14:34:11] CheckForces: Maximum deviation: 0.613 eV/angstrom
[03.04|14:34:11] CheckForces: Actual Threshold
[03.04|14:34:11] CheckForces: # > thr. 2 0 Not OK!
[03.04|14:34:11] CheckForces: MAE 0.187 0.30 OK!
[03.04|14:34:11] CheckForces: R^2 0.832 0.40 OK!
[03.04|14:34:11] CheckForces: --------------------
[03.04|14:34:11]
[03.04|14:34:11] Adding results from step1_attempt5_reference_calc1 to training set
[03.04|14:34:11] Current # training set entries: 27
[03.04|14:34:11] Current # validation set entries: 6
[03.04|14:34:11] Storing data in step1_attempt5_reference_data
[03.04|14:34:11] Deleting step1_attempt4_reference_data
[03.04|14:34:11] Deleting step1_attempt5_reference_calc1
[03.04|14:34:11]
[03.04|14:34:11] Current (cumulative) timings:
[03.04|14:34:11] Time (s) Fraction
[03.04|14:34:11] Ref. calcs 66.08 0.112
[03.04|14:34:11] ML training 415.35 0.705
[03.04|14:34:11] Simulations 107.91 0.183
[03.04|14:34:11]
[03.04|14:34:11]
[03.04|14:34:11]
[03.04|14:34:11] --- Begin summary ---
[03.04|14:34:11] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:34:11] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:34:11] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:34:11] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:34:11] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:34:11] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:34:11] --- End summary ---
[03.04|14:34:11]
[03.04|14:34:11] Running more reference calculations....
[03.04|14:34:11] Running reference calculations on frames [6] from step1_attempt5_simulation/ams.rkf
[03.04|14:34:11] Calculating 1 frames in total
[03.04|14:34:11] Running step1_attempt5_reference_calc2
[03.04|14:34:18] Reference calculations finished!
[03.04|14:34:18] Adding results from step1_attempt5_reference_calc2 to validation set
[03.04|14:34:18] Current # training set entries: 27
[03.04|14:34:18] Current # validation set entries: 7
[03.04|14:34:18] Storing data in step1_attempt5_reference_data
[03.04|14:34:18] Deleting step1_attempt5_reference_calc2
[03.04|14:34:18] Launching reparametrization job: step1_attempt5_training
[03.04|14:34:22] JOB optimizer_001 STARTED
[03.04|14:34:22] JOB optimizer_002 STARTED
[03.04|14:34:22] Starting optimizer_001.prerun()
[03.04|14:34:22] Starting optimizer_002.prerun()
[03.04|14:34:22] optimizer_001.prerun() finished
[03.04|14:34:22] optimizer_002.prerun() finished
[03.04|14:34:22] JOB optimizer_001 RUNNING
[03.04|14:34:22] JOB optimizer_002 RUNNING
[03.04|14:34:22] Executing optimizer_002.run
[03.04|14:34:22] Executing optimizer_001.run
[03.04|14:34:22] Waiting for job optimizer_001 to finish
[03.04|14:35:27] training_set Optimizer: 002 Epoch: 0 Loss: 0.003615
[03.04|14:35:27] validation_set Optimizer: 002 Epoch: 0 Loss: 0.014471
[03.04|14:35:29] training_set Optimizer: 001 Epoch: 0 Loss: 0.004887
[03.04|14:35:29] validation_set Optimizer: 001 Epoch: 0 Loss: 0.022918
[03.04|14:35:31] training_set Optimizer: 002 Epoch: 10 Loss: 0.002727
[03.04|14:35:31] validation_set Optimizer: 002 Epoch: 10 Loss: 0.017962
[03.04|14:35:32] training_set Optimizer: 001 Epoch: 10 Loss: 0.003136
[03.04|14:35:32] validation_set Optimizer: 001 Epoch: 10 Loss: 0.017633
[03.04|14:35:34] training_set Optimizer: 002 Epoch: 20 Loss: 0.003090
[03.04|14:35:34] validation_set Optimizer: 002 Epoch: 20 Loss: 0.015844
[03.04|14:35:36] training_set Optimizer: 001 Epoch: 20 Loss: 0.003035
[03.04|14:35:36] validation_set Optimizer: 001 Epoch: 20 Loss: 0.016217
[03.04|14:35:37] training_set Optimizer: 002 Epoch: 30 Loss: 0.002877
[03.04|14:35:37] validation_set Optimizer: 002 Epoch: 30 Loss: 0.019565
[03.04|14:35:39] training_set Optimizer: 001 Epoch: 30 Loss: 0.003136
[03.04|14:35:39] validation_set Optimizer: 001 Epoch: 30 Loss: 0.018577
[03.04|14:35:41] training_set Optimizer: 002 Epoch: 40 Loss: 0.002686
[03.04|14:35:41] validation_set Optimizer: 002 Epoch: 40 Loss: 0.019450
[03.04|14:35:43] training_set Optimizer: 001 Epoch: 40 Loss: 0.002866
[03.04|14:35:43] validation_set Optimizer: 001 Epoch: 40 Loss: 0.023326
[03.04|14:35:44] training_set Optimizer: 002 Epoch: 50 Loss: 0.002622
[03.04|14:35:44] validation_set Optimizer: 002 Epoch: 50 Loss: 0.019029
[03.04|14:35:46] training_set Optimizer: 001 Epoch: 50 Loss: 0.003309
[03.04|14:35:46] validation_set Optimizer: 001 Epoch: 50 Loss: 0.016461
[03.04|14:35:48] training_set Optimizer: 002 Epoch: 60 Loss: 0.002491
[03.04|14:35:48] validation_set Optimizer: 002 Epoch: 60 Loss: 0.020190
[03.04|14:35:49] training_set Optimizer: 001 Epoch: 60 Loss: 0.002719
[03.04|14:35:49] validation_set Optimizer: 001 Epoch: 60 Loss: 0.018146
[03.04|14:35:51] training_set Optimizer: 002 Epoch: 70 Loss: 0.002690
[03.04|14:35:51] validation_set Optimizer: 002 Epoch: 70 Loss: 0.028543
[03.04|14:35:53] training_set Optimizer: 001 Epoch: 70 Loss: 0.002815
[03.04|14:35:53] validation_set Optimizer: 001 Epoch: 70 Loss: 0.033030
[03.04|14:35:54] training_set Optimizer: 002 Epoch: 80 Loss: 0.002457
[03.04|14:35:54] validation_set Optimizer: 002 Epoch: 80 Loss: 0.016443
[03.04|14:35:56] training_set Optimizer: 001 Epoch: 80 Loss: 0.003065
[03.04|14:35:56] validation_set Optimizer: 001 Epoch: 80 Loss: 0.014944
[03.04|14:35:58] training_set Optimizer: 002 Epoch: 90 Loss: 0.002578
[03.04|14:35:58] validation_set Optimizer: 002 Epoch: 90 Loss: 0.013392
[03.04|14:36:00] training_set Optimizer: 001 Epoch: 90 Loss: 0.002549
[03.04|14:36:00] validation_set Optimizer: 001 Epoch: 90 Loss: 0.029194
[03.04|14:36:01] training_set Optimizer: 002 Epoch: 100 Loss: 0.002252
[03.04|14:36:01] validation_set Optimizer: 002 Epoch: 100 Loss: 0.013098
[03.04|14:36:04] training_set Optimizer: 001 Epoch: 100 Loss: 0.003146
[03.04|14:36:04] validation_set Optimizer: 001 Epoch: 100 Loss: 0.027296
[03.04|14:36:05] training_set Optimizer: 002 Epoch: 110 Loss: 0.002411
[03.04|14:36:05] validation_set Optimizer: 002 Epoch: 110 Loss: 0.015537
[03.04|14:36:07] training_set Optimizer: 001 Epoch: 110 Loss: 0.002689
[03.04|14:36:07] validation_set Optimizer: 001 Epoch: 110 Loss: 0.015707
[03.04|14:36:10] Execution of optimizer_002.run finished with returncode 0
[03.04|14:36:10] JOB optimizer_002 FINISHED
[03.04|14:36:10] Starting optimizer_002.postrun()
[03.04|14:36:10] optimizer_002.postrun() finished
[03.04|14:36:10] JOB optimizer_002 SUCCESSFUL
[03.04|14:36:11] Execution of optimizer_001.run finished with returncode 0
[03.04|14:36:12] JOB optimizer_001 FINISHED
[03.04|14:36:12] Starting optimizer_001.postrun()
[03.04|14:36:12] optimizer_001.postrun() finished
[03.04|14:36:12] JOB optimizer_001 SUCCESSFUL
[03.04|14:36:12] PLAMS environment cleaned up successfully
[03.04|14:36:12] PLAMS run finished. Goodbye
[03.04|14:36:12] ParAMSResults
[03.04|14:36:12] Newly created parameter file/dir: step1_attempt5_training/results/optimization/optimizer_001/m3gnet
[03.04|14:36:12] Newly created parameter file/dir: step1_attempt5_training/results/optimization/optimizer_002/m3gnet
[03.04|14:36:12] Done!
[03.04|14:36:12] Deleting step1_attempt4_training
[03.04|14:36:12] ##########################
[03.04|14:36:12] ### Step 1 / Attempt 6 ###
[03.04|14:36:12] ##########################
[03.04|14:36:12] MD Steps: 10 (cumulative: 10)
[03.04|14:36:12] Current engine settings:
[03.04|14:36:12]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt5_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt5_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:36:12] Running step1_attempt6_simulation...
[03.04|14:36:38] Job step1_attempt6_simulation finished
[03.04|14:36:38] Deleting files that are no longer needed...
[03.04|14:36:38] Energy uncertainty for final frame of step1_attempt6_simulation: 0.0871 eV
[03.04|14:36:38] 0.0073 eV/atom
[03.04|14:36:38] Forces uncertainty for final frame of step1_attempt6_simulation: 0.1288 eV/angstrom
[03.04|14:36:38] Launching reference calculation
[03.04|14:36:45] Reference calculation finished!
[03.04|14:36:45] Checking success for step1_attempt6
[03.04|14:37:04] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:37:04] CheckEnergy: normalization coefficient = 12
[03.04|14:37:04] CheckEnergy: Actual Threshold
[03.04|14:37:04] CheckEnergy: dE/12 0.0038 0.2000 OK!
[03.04|14:37:04] CheckEnergy: ddE/12 0.0049 0.0050 OK! (relative to step1_attempt5_simulation:MDStep10)
[03.04|14:37:04]
[03.04|14:37:04] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:37:04] CheckForces: ------------
[03.04|14:37:04] CheckForces: Reference job from step1_attempt6_reference_calc1
[03.04|14:37:04] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt6_simulation
[03.04|14:37:04] CheckForces: ------------
[03.04|14:37:04] CheckForces: Histogram of forces
[03.04|14:37:04] CheckForces: eV/Ang Ref Pred
[03.04|14:37:04] CheckForces: -3 0 0
[03.04|14:37:04] CheckForces: -2 4 4
[03.04|14:37:04] CheckForces: -1 15 14
[03.04|14:37:04] CheckForces: 0 15 17
[03.04|14:37:04] CheckForces: 1 2 1
[03.04|14:37:04] CheckForces: 2 0 0
[03.04|14:37:04] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:37:04] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:37:04] CheckForces: Ref Pred Delta Threshold
[03.04|14:37:04] CheckForces: -0.31 0.26 0.56 0.51
[03.04|14:37:04] CheckForces: Maximum deviation: 0.564 eV/angstrom
[03.04|14:37:04] CheckForces: Actual Threshold
[03.04|14:37:04] CheckForces: # > thr. 1 0 Not OK!
[03.04|14:37:04] CheckForces: MAE 0.162 0.30 OK!
[03.04|14:37:04] CheckForces: R^2 0.916 0.40 OK!
[03.04|14:37:04] CheckForces: --------------------
[03.04|14:37:04]
[03.04|14:37:04] Adding results from step1_attempt6_reference_calc1 to training set
[03.04|14:37:04] Current # training set entries: 28
[03.04|14:37:04] Current # validation set entries: 7
[03.04|14:37:04] Storing data in step1_attempt6_reference_data
[03.04|14:37:04] Deleting step1_attempt5_reference_data
[03.04|14:37:04] Deleting step1_attempt6_reference_calc1
[03.04|14:37:04]
[03.04|14:37:04] Current (cumulative) timings:
[03.04|14:37:04] Time (s) Fraction
[03.04|14:37:04] Ref. calcs 79.72 0.107
[03.04|14:37:04] ML training 529.81 0.713
[03.04|14:37:04] Simulations 133.38 0.180
[03.04|14:37:04]
[03.04|14:37:04]
[03.04|14:37:04]
[03.04|14:37:04] --- Begin summary ---
[03.04|14:37:04] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:37:04] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:37:04] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:37:04] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:37:04] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:37:04] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:37:04] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:37:04] --- End summary ---
[03.04|14:37:04]
[03.04|14:37:04] Running more reference calculations....
[03.04|14:37:04] Running reference calculations on frames [6] from step1_attempt6_simulation/ams.rkf
[03.04|14:37:04] Calculating 1 frames in total
[03.04|14:37:04] Running step1_attempt6_reference_calc2
[03.04|14:37:11] Reference calculations finished!
[03.04|14:37:11] Adding results from step1_attempt6_reference_calc2 to validation set
[03.04|14:37:11] Current # training set entries: 28
[03.04|14:37:11] Current # validation set entries: 8
[03.04|14:37:11] Storing data in step1_attempt6_reference_data
[03.04|14:37:11] Deleting step1_attempt6_reference_calc2
[03.04|14:37:11] Launching reparametrization job: step1_attempt6_training
[03.04|14:37:15] JOB optimizer_001 STARTED
[03.04|14:37:15] JOB optimizer_002 STARTED
[03.04|14:37:15] Starting optimizer_001.prerun()
[03.04|14:37:15] optimizer_001.prerun() finished
[03.04|14:37:15] Starting optimizer_002.prerun()
[03.04|14:37:15] optimizer_002.prerun() finished
[03.04|14:37:15] JOB optimizer_001 RUNNING
[03.04|14:37:15] Executing optimizer_001.run
[03.04|14:37:15] JOB optimizer_002 RUNNING
[03.04|14:37:15] Executing optimizer_002.run
[03.04|14:37:15] Waiting for job optimizer_001 to finish
[03.04|14:38:23] training_set Optimizer: 002 Epoch: 0 Loss: 0.003818
[03.04|14:38:23] validation_set Optimizer: 002 Epoch: 0 Loss: 0.017846
[03.04|14:38:24] training_set Optimizer: 001 Epoch: 0 Loss: 0.004759
[03.04|14:38:24] validation_set Optimizer: 001 Epoch: 0 Loss: 0.024937
[03.04|14:38:26] training_set Optimizer: 002 Epoch: 10 Loss: 0.002245
[03.04|14:38:26] validation_set Optimizer: 002 Epoch: 10 Loss: 0.016218
[03.04|14:38:27] training_set Optimizer: 001 Epoch: 10 Loss: 0.002943
[03.04|14:38:27] validation_set Optimizer: 001 Epoch: 10 Loss: 0.017649
[03.04|14:38:30] training_set Optimizer: 002 Epoch: 20 Loss: 0.002310
[03.04|14:38:30] validation_set Optimizer: 002 Epoch: 20 Loss: 0.014807
[03.04|14:38:31] training_set Optimizer: 001 Epoch: 20 Loss: 0.003019
[03.04|14:38:31] validation_set Optimizer: 001 Epoch: 20 Loss: 0.018172
[03.04|14:38:33] training_set Optimizer: 002 Epoch: 30 Loss: 0.002237
[03.04|14:38:33] validation_set Optimizer: 002 Epoch: 30 Loss: 0.015824
[03.04|14:38:34] training_set Optimizer: 001 Epoch: 30 Loss: 0.002594
[03.04|14:38:34] validation_set Optimizer: 001 Epoch: 30 Loss: 0.017998
[03.04|14:38:37] training_set Optimizer: 002 Epoch: 40 Loss: 0.002241
[03.04|14:38:37] validation_set Optimizer: 002 Epoch: 40 Loss: 0.012582
[03.04|14:38:37] training_set Optimizer: 001 Epoch: 40 Loss: 0.002433
[03.04|14:38:37] validation_set Optimizer: 001 Epoch: 40 Loss: 0.015567
[03.04|14:38:40] training_set Optimizer: 002 Epoch: 50 Loss: 0.002343
[03.04|14:38:40] validation_set Optimizer: 002 Epoch: 50 Loss: 0.013622
[03.04|14:38:41] training_set Optimizer: 001 Epoch: 50 Loss: 0.002549
[03.04|14:38:41] validation_set Optimizer: 001 Epoch: 50 Loss: 0.014805
[03.04|14:38:43] training_set Optimizer: 002 Epoch: 60 Loss: 0.002132
[03.04|14:38:43] validation_set Optimizer: 002 Epoch: 60 Loss: 0.016372
[03.04|14:38:44] training_set Optimizer: 001 Epoch: 60 Loss: 0.002385
[03.04|14:38:44] validation_set Optimizer: 001 Epoch: 60 Loss: 0.014510
[03.04|14:38:47] training_set Optimizer: 002 Epoch: 70 Loss: 0.002210
[03.04|14:38:47] validation_set Optimizer: 002 Epoch: 70 Loss: 0.020349
[03.04|14:38:48] training_set Optimizer: 001 Epoch: 70 Loss: 0.002602
[03.04|14:38:48] validation_set Optimizer: 001 Epoch: 70 Loss: 0.014227
[03.04|14:38:50] training_set Optimizer: 002 Epoch: 80 Loss: 0.002054
[03.04|14:38:50] validation_set Optimizer: 002 Epoch: 80 Loss: 0.014082
[03.04|14:38:51] training_set Optimizer: 001 Epoch: 80 Loss: 0.002415
[03.04|14:38:51] validation_set Optimizer: 001 Epoch: 80 Loss: 0.013184
[03.04|14:38:54] training_set Optimizer: 002 Epoch: 90 Loss: 0.002038
[03.04|14:38:54] validation_set Optimizer: 002 Epoch: 90 Loss: 0.016147
[03.04|14:38:54] training_set Optimizer: 001 Epoch: 90 Loss: 0.002454
[03.04|14:38:54] validation_set Optimizer: 001 Epoch: 90 Loss: 0.025039
[03.04|14:38:57] training_set Optimizer: 002 Epoch: 100 Loss: 0.001978
[03.04|14:38:57] validation_set Optimizer: 002 Epoch: 100 Loss: 0.012958
[03.04|14:38:58] training_set Optimizer: 001 Epoch: 100 Loss: 0.002363
[03.04|14:38:58] validation_set Optimizer: 001 Epoch: 100 Loss: 0.019068
[03.04|14:39:00] training_set Optimizer: 002 Epoch: 110 Loss: 0.002253
[03.04|14:39:00] validation_set Optimizer: 002 Epoch: 110 Loss: 0.012380
[03.04|14:39:01] training_set Optimizer: 001 Epoch: 110 Loss: 0.002342
[03.04|14:39:01] validation_set Optimizer: 001 Epoch: 110 Loss: 0.014562
[03.04|14:39:06] Execution of optimizer_002.run finished with returncode 0
[03.04|14:39:06] JOB optimizer_002 FINISHED
[03.04|14:39:06] Starting optimizer_002.postrun()
[03.04|14:39:06] optimizer_002.postrun() finished
[03.04|14:39:06] JOB optimizer_002 SUCCESSFUL
[03.04|14:39:06] Execution of optimizer_001.run finished with returncode 0
[03.04|14:39:06] JOB optimizer_001 FINISHED
[03.04|14:39:06] Starting optimizer_001.postrun()
[03.04|14:39:06] optimizer_001.postrun() finished
[03.04|14:39:07] JOB optimizer_001 SUCCESSFUL
[03.04|14:39:07] PLAMS environment cleaned up successfully
[03.04|14:39:07] PLAMS run finished. Goodbye
[03.04|14:39:07] ParAMSResults
[03.04|14:39:07] Newly created parameter file/dir: step1_attempt6_training/results/optimization/optimizer_001/m3gnet
[03.04|14:39:07] Newly created parameter file/dir: step1_attempt6_training/results/optimization/optimizer_002/m3gnet
[03.04|14:39:07] Done!
[03.04|14:39:07] Deleting step1_attempt5_training
[03.04|14:39:07] ##########################
[03.04|14:39:07] ### Step 1 / Attempt 7 ###
[03.04|14:39:07] ##########################
[03.04|14:39:07] MD Steps: 10 (cumulative: 10)
[03.04|14:39:07] Current engine settings:
[03.04|14:39:07]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt6_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt6_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:39:07] Running step1_attempt7_simulation...
[03.04|14:39:33] Job step1_attempt7_simulation finished
[03.04|14:39:33] Deleting files that are no longer needed...
[03.04|14:39:33] Energy uncertainty for final frame of step1_attempt7_simulation: 0.0754 eV
[03.04|14:39:33] 0.0063 eV/atom
[03.04|14:39:33] Forces uncertainty for final frame of step1_attempt7_simulation: 0.2579 eV/angstrom
[03.04|14:39:34] Launching reference calculation
[03.04|14:39:42] Reference calculation finished!
[03.04|14:39:42] Checking success for step1_attempt7
[03.04|14:40:02] CheckEnergy: Checking energy for MDStep10, n_atoms = 12
[03.04|14:40:02] CheckEnergy: normalization coefficient = 12
[03.04|14:40:02] CheckEnergy: Actual Threshold
[03.04|14:40:02] CheckEnergy: dE/12 0.0084 0.2000 OK!
[03.04|14:40:02] CheckEnergy: ddE/12 -0.0016 0.0050 OK! (relative to step1_attempt6_simulation:MDStep10)
[03.04|14:40:02]
[03.04|14:40:02] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:40:02] CheckForces: ------------
[03.04|14:40:02] CheckForces: Reference job from step1_attempt7_reference_calc1
[03.04|14:40:02] CheckForces: Prediction job from final frame (MDStep10) of step1_attempt7_simulation
[03.04|14:40:02] CheckForces: ------------
[03.04|14:40:02] CheckForces: Histogram of forces
[03.04|14:40:02] CheckForces: eV/Ang Ref Pred
[03.04|14:40:02] CheckForces: -2 2 1
[03.04|14:40:02] CheckForces: -1 17 20
[03.04|14:40:02] CheckForces: 0 15 13
[03.04|14:40:02] CheckForces: 1 1 1
[03.04|14:40:02] CheckForces: 2 1 1
[03.04|14:40:02] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:40:02] CheckForces: All force components are within the acceptable error!
[03.04|14:40:02] CheckForces: Maximum deviation: 0.424 eV/angstrom
[03.04|14:40:02] CheckForces: Actual Threshold
[03.04|14:40:02] CheckForces: # > thr. 0 0 OK!
[03.04|14:40:02] CheckForces: MAE 0.134 0.30 OK!
[03.04|14:40:02] CheckForces: R^2 0.919 0.40 OK!
[03.04|14:40:02] CheckForces: --------------------
[03.04|14:40:02]
[03.04|14:40:02] Adding results from step1_attempt7_reference_calc1 to training set
[03.04|14:40:02] Current # training set entries: 29
[03.04|14:40:02] Current # validation set entries: 8
[03.04|14:40:02] Storing data in step1_attempt7_reference_data
[03.04|14:40:02] Deleting step1_attempt6_reference_data
[03.04|14:40:02] Deleting step1_attempt7_reference_calc1
[03.04|14:40:02]
[03.04|14:40:02] Current (cumulative) timings:
[03.04|14:40:02] Time (s) Fraction
[03.04|14:40:02] Ref. calcs 95.13 0.106
[03.04|14:40:02] ML training 645.84 0.717
[03.04|14:40:02] Simulations 159.52 0.177
[03.04|14:40:02]
[03.04|14:40:02]
[03.04|14:40:02] Step 1 finished successfully!
[03.04|14:40:02]
[03.04|14:40:02] --- Begin summary ---
[03.04|14:40:02] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:40:02] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:40:02] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:40:02] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:40:02] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:40:02] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:40:02] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:40:02] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:40:02] --- End summary ---
[03.04|14:40:02]
[03.04|14:40:02] ##########################
[03.04|14:40:02] ### Step 2 / Attempt 1 ###
[03.04|14:40:02] ##########################
[03.04|14:40:02] MD Steps: 31 (cumulative: 41)
[03.04|14:40:02] Current engine settings:
[03.04|14:40:02]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt6_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step1_attempt6_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:40:02] Running step2_attempt1_simulation...
[03.04|14:40:23] Job step2_attempt1_simulation finished
[03.04|14:40:23] Deleting files that are no longer needed...
[03.04|14:40:23] Deleting step1_attempt1_simulation
[03.04|14:40:23] Deleting step1_attempt2_simulation
[03.04|14:40:23] Deleting step1_attempt3_simulation
[03.04|14:40:23] Deleting step1_attempt4_simulation
[03.04|14:40:23] Deleting step1_attempt5_simulation
[03.04|14:40:23] Deleting step1_attempt6_simulation
[03.04|14:40:23] Energy uncertainty for final frame of step2_attempt1_simulation: 0.0302 eV
[03.04|14:40:23] 0.0025 eV/atom
[03.04|14:40:23] Forces uncertainty for final frame of step2_attempt1_simulation: 0.4215 eV/angstrom
[03.04|14:40:23] Launching reference calculation
[03.04|14:40:32] Reference calculation finished!
[03.04|14:40:32] Checking success for step2_attempt1
[03.04|14:40:52] CheckEnergy: Checking energy for MDStep41, n_atoms = 12
[03.04|14:40:52] CheckEnergy: normalization coefficient = 12
[03.04|14:40:52] CheckEnergy: Actual Threshold
[03.04|14:40:52] CheckEnergy: dE/12 0.0083 0.2000 OK!
[03.04|14:40:52] CheckEnergy: ddE/12 -0.0001 0.0050 OK! (relative to step1_attempt7_simulation:MDStep10)
[03.04|14:40:52]
[03.04|14:40:52] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:40:52] CheckForces: ------------
[03.04|14:40:52] CheckForces: Reference job from step2_attempt1_reference_calc1
[03.04|14:40:52] CheckForces: Prediction job from final frame (MDStep41) of step2_attempt1_simulation
[03.04|14:40:52] CheckForces: ------------
[03.04|14:40:52] CheckForces: Histogram of forces
[03.04|14:40:52] CheckForces: eV/Ang Ref Pred
[03.04|14:40:52] CheckForces: -2 0 1
[03.04|14:40:52] CheckForces: -1 20 17
[03.04|14:40:52] CheckForces: 0 15 17
[03.04|14:40:52] CheckForces: 1 1 1
[03.04|14:40:52] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:40:52] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:40:52] CheckForces: Ref Pred Delta Threshold
[03.04|14:40:52] CheckForces: 0.91 -0.35 1.26 0.55
[03.04|14:40:52] CheckForces: 0.32 1.01 0.69 0.51
[03.04|14:40:52] CheckForces: -0.90 0.65 1.55 0.55
[03.04|14:40:52] CheckForces: -0.06 0.69 0.74 0.50
[03.04|14:40:52] CheckForces: -0.22 0.44 0.66 0.51
[03.04|14:40:52] CheckForces: 0.44 -0.10 0.54 0.52
[03.04|14:40:52] CheckForces: ... and 3 more.
[03.04|14:40:52] CheckForces: Maximum deviation: 1.551 eV/angstrom
[03.04|14:40:52] CheckForces: Actual Threshold
[03.04|14:40:52] CheckForces: # > thr. 9 0 Not OK!
[03.04|14:40:52] CheckForces: MAE 0.370 0.30 Not OK!
[03.04|14:40:52] CheckForces: R^2 0.174 0.40 Not OK!
[03.04|14:40:52] CheckForces: --------------------
[03.04|14:40:52]
[03.04|14:40:52] Adding results from step2_attempt1_reference_calc1 to training set
[03.04|14:40:52] Current # training set entries: 30
[03.04|14:40:52] Current # validation set entries: 8
[03.04|14:40:52] Storing data in step2_attempt1_reference_data
[03.04|14:40:52] Deleting step1_attempt7_reference_data
[03.04|14:40:52] Deleting step2_attempt1_reference_calc1
[03.04|14:40:52]
[03.04|14:40:52] Current (cumulative) timings:
[03.04|14:40:52] Time (s) Fraction
[03.04|14:40:52] Ref. calcs 104.03 0.112
[03.04|14:40:52] ML training 645.84 0.694
[03.04|14:40:52] Simulations 180.35 0.194
[03.04|14:40:52]
[03.04|14:40:52]
[03.04|14:40:52]
[03.04|14:40:52] --- Begin summary ---
[03.04|14:40:52] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:40:52] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:40:52] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:40:52] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:40:52] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:40:52] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:40:52] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:40:52] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:40:52] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:40:52] --- End summary ---
[03.04|14:40:52]
[03.04|14:40:52] Running more reference calculations....
[03.04|14:40:52] Running reference calculations on frames [16] from step2_attempt1_simulation/ams.rkf
[03.04|14:40:52] Calculating 1 frames in total
[03.04|14:40:52] Running step2_attempt1_reference_calc2
[03.04|14:41:01] Reference calculations finished!
[03.04|14:41:01] Adding results from step2_attempt1_reference_calc2 to validation set
[03.04|14:41:01] Current # training set entries: 30
[03.04|14:41:01] Current # validation set entries: 9
[03.04|14:41:01] Storing data in step2_attempt1_reference_data
[03.04|14:41:02] Deleting step2_attempt1_reference_calc2
[03.04|14:41:02] Launching reparametrization job: step2_attempt1_training
[03.04|14:41:06] JOB optimizer_001 STARTED
[03.04|14:41:06] JOB optimizer_002 STARTED
[03.04|14:41:06] Starting optimizer_001.prerun()
[03.04|14:41:06] Starting optimizer_002.prerun()
[03.04|14:41:06] optimizer_001.prerun() finished
[03.04|14:41:06] optimizer_002.prerun() finished
[03.04|14:41:06] JOB optimizer_001 RUNNING
[03.04|14:41:06] Executing optimizer_001.run
[03.04|14:41:06] JOB optimizer_002 RUNNING
[03.04|14:41:06] Executing optimizer_002.run
[03.04|14:41:06] Waiting for job optimizer_001 to finish
[03.04|14:41:54] training_set Optimizer: 001 Epoch: 0 Loss: 0.006905
[03.04|14:41:54] validation_set Optimizer: 001 Epoch: 0 Loss: 0.043130
[03.04|14:41:54] training_set Optimizer: 002 Epoch: 0 Loss: 0.004854
[03.04|14:41:54] validation_set Optimizer: 002 Epoch: 0 Loss: 0.017326
[03.04|14:41:57] training_set Optimizer: 001 Epoch: 10 Loss: 0.002495
[03.04|14:41:57] validation_set Optimizer: 001 Epoch: 10 Loss: 0.014843
[03.04|14:41:57] training_set Optimizer: 002 Epoch: 10 Loss: 0.002903
[03.04|14:41:57] validation_set Optimizer: 002 Epoch: 10 Loss: 0.017489
[03.04|14:42:00] training_set Optimizer: 001 Epoch: 20 Loss: 0.002437
[03.04|14:42:00] validation_set Optimizer: 001 Epoch: 20 Loss: 0.013231
[03.04|14:42:00] training_set Optimizer: 002 Epoch: 20 Loss: 0.002682
[03.04|14:42:00] validation_set Optimizer: 002 Epoch: 20 Loss: 0.020204
[03.04|14:42:04] training_set Optimizer: 001 Epoch: 30 Loss: 0.002485
[03.04|14:42:04] validation_set Optimizer: 001 Epoch: 30 Loss: 0.013214
[03.04|14:42:04] training_set Optimizer: 002 Epoch: 30 Loss: 0.002482
[03.04|14:42:04] validation_set Optimizer: 002 Epoch: 30 Loss: 0.015626
[03.04|14:42:07] training_set Optimizer: 002 Epoch: 40 Loss: 0.002502
[03.04|14:42:07] validation_set Optimizer: 002 Epoch: 40 Loss: 0.015133
[03.04|14:42:07] training_set Optimizer: 001 Epoch: 40 Loss: 0.002338
[03.04|14:42:07] validation_set Optimizer: 001 Epoch: 40 Loss: 0.015476
[03.04|14:42:10] training_set Optimizer: 001 Epoch: 50 Loss: 0.002311
[03.04|14:42:10] validation_set Optimizer: 001 Epoch: 50 Loss: 0.013612
[03.04|14:42:10] training_set Optimizer: 002 Epoch: 50 Loss: 0.002466
[03.04|14:42:10] validation_set Optimizer: 002 Epoch: 50 Loss: 0.020893
[03.04|14:42:14] training_set Optimizer: 001 Epoch: 60 Loss: 0.002474
[03.04|14:42:14] validation_set Optimizer: 001 Epoch: 60 Loss: 0.017077
[03.04|14:42:14] training_set Optimizer: 002 Epoch: 60 Loss: 0.002333
[03.04|14:42:14] validation_set Optimizer: 002 Epoch: 60 Loss: 0.015182
[03.04|14:42:17] training_set Optimizer: 001 Epoch: 70 Loss: 0.002203
[03.04|14:42:17] validation_set Optimizer: 001 Epoch: 70 Loss: 0.012294
[03.04|14:42:17] training_set Optimizer: 002 Epoch: 70 Loss: 0.002236
[03.04|14:42:17] validation_set Optimizer: 002 Epoch: 70 Loss: 0.017534
[03.04|14:42:20] training_set Optimizer: 002 Epoch: 80 Loss: 0.002186
[03.04|14:42:20] validation_set Optimizer: 002 Epoch: 80 Loss: 0.019403
[03.04|14:42:21] training_set Optimizer: 001 Epoch: 80 Loss: 0.002186
[03.04|14:42:21] validation_set Optimizer: 001 Epoch: 80 Loss: 0.011126
[03.04|14:42:24] training_set Optimizer: 001 Epoch: 90 Loss: 0.002336
[03.04|14:42:24] validation_set Optimizer: 001 Epoch: 90 Loss: 0.012947
[03.04|14:42:24] training_set Optimizer: 002 Epoch: 90 Loss: 0.002474
[03.04|14:42:24] validation_set Optimizer: 002 Epoch: 90 Loss: 0.020130
[03.04|14:42:27] training_set Optimizer: 001 Epoch: 100 Loss: 0.002187
[03.04|14:42:27] validation_set Optimizer: 001 Epoch: 100 Loss: 0.011044
[03.04|14:42:27] training_set Optimizer: 002 Epoch: 100 Loss: 0.002186
[03.04|14:42:27] validation_set Optimizer: 002 Epoch: 100 Loss: 0.012596
[03.04|14:42:31] training_set Optimizer: 001 Epoch: 110 Loss: 0.002052
[03.04|14:42:31] validation_set Optimizer: 001 Epoch: 110 Loss: 0.014279
[03.04|14:42:31] training_set Optimizer: 002 Epoch: 110 Loss: 0.002124
[03.04|14:42:31] validation_set Optimizer: 002 Epoch: 110 Loss: 0.013178
[03.04|14:42:35] Execution of optimizer_002.run finished with returncode 0
[03.04|14:42:35] Execution of optimizer_001.run finished with returncode 0
[03.04|14:42:35] JOB optimizer_001 FINISHED
[03.04|14:42:35] Starting optimizer_001.postrun()
[03.04|14:42:35] optimizer_001.postrun() finished
[03.04|14:42:35] JOB optimizer_002 FINISHED
[03.04|14:42:36] Starting optimizer_002.postrun()
[03.04|14:42:36] optimizer_002.postrun() finished
[03.04|14:42:36] JOB optimizer_002 SUCCESSFUL
[03.04|14:42:36] JOB optimizer_001 SUCCESSFUL
[03.04|14:42:36] PLAMS environment cleaned up successfully
[03.04|14:42:36] PLAMS run finished. Goodbye
[03.04|14:42:36] ParAMSResults
[03.04|14:42:36] Newly created parameter file/dir: step2_attempt1_training/results/optimization/optimizer_001/m3gnet
[03.04|14:42:36] Newly created parameter file/dir: step2_attempt1_training/results/optimization/optimizer_002/m3gnet
[03.04|14:42:36] Done!
[03.04|14:42:36] Deleting step1_attempt6_training
[03.04|14:42:36] ##########################
[03.04|14:42:36] ### Step 2 / Attempt 2 ###
[03.04|14:42:36] ##########################
[03.04|14:42:36] MD Steps: 31 (cumulative: 41)
[03.04|14:42:36] Current engine settings:
[03.04|14:42:36]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt1_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt1_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:42:36] Running step2_attempt2_simulation...
[03.04|14:42:57] Job step2_attempt2_simulation finished
[03.04|14:42:57] Deleting files that are no longer needed...
[03.04|14:42:57] Energy uncertainty for final frame of step2_attempt2_simulation: 0.0400 eV
[03.04|14:42:57] 0.0033 eV/atom
[03.04|14:42:57] Forces uncertainty for final frame of step2_attempt2_simulation: 0.2573 eV/angstrom
[03.04|14:42:57] Launching reference calculation
[03.04|14:43:07] Reference calculation finished!
[03.04|14:43:07] Checking success for step2_attempt2
[03.04|14:43:26] CheckEnergy: Checking energy for MDStep41, n_atoms = 12
[03.04|14:43:26] CheckEnergy: normalization coefficient = 12
[03.04|14:43:26] CheckEnergy: Actual Threshold
[03.04|14:43:26] CheckEnergy: dE/12 -0.0027 0.2000 OK!
[03.04|14:43:26] CheckEnergy: ddE/12 0.0039 0.0050 OK! (relative to step2_attempt1_simulation:MDStep41)
[03.04|14:43:26]
[03.04|14:43:26] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:43:26] CheckForces: ------------
[03.04|14:43:26] CheckForces: Reference job from step2_attempt2_reference_calc1
[03.04|14:43:26] CheckForces: Prediction job from final frame (MDStep41) of step2_attempt2_simulation
[03.04|14:43:26] CheckForces: ------------
[03.04|14:43:26] CheckForces: Histogram of forces
[03.04|14:43:26] CheckForces: eV/Ang Ref Pred
[03.04|14:43:26] CheckForces: -2 0 0
[03.04|14:43:26] CheckForces: -1 19 19
[03.04|14:43:26] CheckForces: 0 16 16
[03.04|14:43:26] CheckForces: 1 1 1
[03.04|14:43:26] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:43:26] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:43:26] CheckForces: Ref Pred Delta Threshold
[03.04|14:43:26] CheckForces: -0.39 0.38 0.77 0.52
[03.04|14:43:26] CheckForces: -0.08 -0.78 0.70 0.50
[03.04|14:43:26] CheckForces: Maximum deviation: 0.768 eV/angstrom
[03.04|14:43:26] CheckForces: Actual Threshold
[03.04|14:43:26] CheckForces: # > thr. 2 0 Not OK!
[03.04|14:43:26] CheckForces: MAE 0.202 0.30 OK!
[03.04|14:43:26] CheckForces: R^2 0.638 0.40 OK!
[03.04|14:43:26] CheckForces: --------------------
[03.04|14:43:26]
[03.04|14:43:26] Adding results from step2_attempt2_reference_calc1 to training set
[03.04|14:43:26] Current # training set entries: 31
[03.04|14:43:26] Current # validation set entries: 9
[03.04|14:43:26] Storing data in step2_attempt2_reference_data
[03.04|14:43:26] Deleting step2_attempt1_reference_data
[03.04|14:43:26] Deleting step2_attempt2_reference_calc1
[03.04|14:43:26]
[03.04|14:43:26] Current (cumulative) timings:
[03.04|14:43:26] Time (s) Fraction
[03.04|14:43:26] Ref. calcs 123.04 0.116
[03.04|14:43:26] ML training 740.44 0.696
[03.04|14:43:26] Simulations 200.95 0.189
[03.04|14:43:26]
[03.04|14:43:26]
[03.04|14:43:26]
[03.04|14:43:26] --- Begin summary ---
[03.04|14:43:26] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:43:26] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:43:26] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:43:26] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:43:26] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:43:26] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:43:26] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:43:26] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:43:26] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:43:26] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:43:26] --- End summary ---
[03.04|14:43:26]
[03.04|14:43:26] Running more reference calculations....
[03.04|14:43:26] Running reference calculations on frames [16] from step2_attempt2_simulation/ams.rkf
[03.04|14:43:26] Calculating 1 frames in total
[03.04|14:43:26] Running step2_attempt2_reference_calc2
[03.04|14:43:35] Reference calculations finished!
[03.04|14:43:35] Adding results from step2_attempt2_reference_calc2 to validation set
[03.04|14:43:35] Current # training set entries: 31
[03.04|14:43:35] Current # validation set entries: 10
[03.04|14:43:35] Storing data in step2_attempt2_reference_data
[03.04|14:43:35] Deleting step2_attempt2_reference_calc2
[03.04|14:43:35] Launching reparametrization job: step2_attempt2_training
[03.04|14:43:39] JOB optimizer_001 STARTED
[03.04|14:43:39] JOB optimizer_002 STARTED
[03.04|14:43:39] Starting optimizer_001.prerun()
[03.04|14:43:39] Starting optimizer_002.prerun()
[03.04|14:43:39] optimizer_001.prerun() finished
[03.04|14:43:39] optimizer_002.prerun() finished
[03.04|14:43:39] JOB optimizer_002 RUNNING
[03.04|14:43:39] Executing optimizer_002.run
[03.04|14:43:39] JOB optimizer_001 RUNNING
[03.04|14:43:39] Executing optimizer_001.run
[03.04|14:43:39] Waiting for job optimizer_001 to finish
[03.04|14:44:45] training_set Optimizer: 002 Epoch: 0 Loss: 0.003544
[03.04|14:44:45] validation_set Optimizer: 002 Epoch: 0 Loss: 0.022651
[03.04|14:44:49] training_set Optimizer: 002 Epoch: 10 Loss: 0.002054
[03.04|14:44:49] validation_set Optimizer: 002 Epoch: 10 Loss: 0.019172
[03.04|14:44:52] training_set Optimizer: 002 Epoch: 20 Loss: 0.002434
[03.04|14:44:52] validation_set Optimizer: 002 Epoch: 20 Loss: 0.020761
[03.04|14:44:55] training_set Optimizer: 002 Epoch: 30 Loss: 0.002110
[03.04|14:44:55] validation_set Optimizer: 002 Epoch: 30 Loss: 0.018273
[03.04|14:44:59] training_set Optimizer: 002 Epoch: 40 Loss: 0.001946
[03.04|14:44:59] validation_set Optimizer: 002 Epoch: 40 Loss: 0.012572
[03.04|14:45:02] training_set Optimizer: 002 Epoch: 50 Loss: 0.002080
[03.04|14:45:02] validation_set Optimizer: 002 Epoch: 50 Loss: 0.019872
[03.04|14:45:03] training_set Optimizer: 001 Epoch: 0 Loss: 0.005018
[03.04|14:45:03] validation_set Optimizer: 001 Epoch: 0 Loss: 0.012376
[03.04|14:45:06] training_set Optimizer: 002 Epoch: 60 Loss: 0.002083
[03.04|14:45:06] validation_set Optimizer: 002 Epoch: 60 Loss: 0.011039
[03.04|14:45:07] training_set Optimizer: 001 Epoch: 10 Loss: 0.002151
[03.04|14:45:07] validation_set Optimizer: 001 Epoch: 10 Loss: 0.011625
[03.04|14:45:10] training_set Optimizer: 002 Epoch: 70 Loss: 0.002081
[03.04|14:45:10] validation_set Optimizer: 002 Epoch: 70 Loss: 0.024989
[03.04|14:45:11] training_set Optimizer: 001 Epoch: 20 Loss: 0.002024
[03.04|14:45:11] validation_set Optimizer: 001 Epoch: 20 Loss: 0.019735
[03.04|14:45:14] training_set Optimizer: 002 Epoch: 80 Loss: 0.001942
[03.04|14:45:14] validation_set Optimizer: 002 Epoch: 80 Loss: 0.011909
[03.04|14:45:15] training_set Optimizer: 001 Epoch: 30 Loss: 0.002001
[03.04|14:45:15] validation_set Optimizer: 001 Epoch: 30 Loss: 0.025674
[03.04|14:45:18] training_set Optimizer: 002 Epoch: 90 Loss: 0.001888
[03.04|14:45:18] validation_set Optimizer: 002 Epoch: 90 Loss: 0.016022
[03.04|14:45:18] training_set Optimizer: 001 Epoch: 40 Loss: 0.002089
[03.04|14:45:18] validation_set Optimizer: 001 Epoch: 40 Loss: 0.016492
[03.04|14:45:21] training_set Optimizer: 002 Epoch: 100 Loss: 0.002148
[03.04|14:45:21] validation_set Optimizer: 002 Epoch: 100 Loss: 0.014175
[03.04|14:45:22] training_set Optimizer: 001 Epoch: 50 Loss: 0.003662
[03.04|14:45:22] validation_set Optimizer: 001 Epoch: 50 Loss: 0.014580
[03.04|14:45:25] training_set Optimizer: 002 Epoch: 110 Loss: 0.001932
[03.04|14:45:25] validation_set Optimizer: 002 Epoch: 110 Loss: 0.016939
[03.04|14:45:26] training_set Optimizer: 001 Epoch: 60 Loss: 0.001973
[03.04|14:45:26] validation_set Optimizer: 001 Epoch: 60 Loss: 0.009576
[03.04|14:45:30] training_set Optimizer: 001 Epoch: 70 Loss: 0.002710
[03.04|14:45:30] validation_set Optimizer: 001 Epoch: 70 Loss: 0.020164
[03.04|14:45:31] Execution of optimizer_002.run finished with returncode 0
[03.04|14:45:31] JOB optimizer_002 FINISHED
[03.04|14:45:31] Starting optimizer_002.postrun()
[03.04|14:45:31] optimizer_002.postrun() finished
[03.04|14:45:31] JOB optimizer_002 SUCCESSFUL
[03.04|14:45:33] training_set Optimizer: 001 Epoch: 80 Loss: 0.002331
[03.04|14:45:33] validation_set Optimizer: 001 Epoch: 80 Loss: 0.013185
[03.04|14:45:37] training_set Optimizer: 001 Epoch: 90 Loss: 0.001974
[03.04|14:45:37] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013782
[03.04|14:45:40] training_set Optimizer: 001 Epoch: 100 Loss: 0.002082
[03.04|14:45:40] validation_set Optimizer: 001 Epoch: 100 Loss: 0.011300
[03.04|14:45:43] training_set Optimizer: 001 Epoch: 110 Loss: 0.001997
[03.04|14:45:43] validation_set Optimizer: 001 Epoch: 110 Loss: 0.036581
[03.04|14:45:48] Execution of optimizer_001.run finished with returncode 0
[03.04|14:45:48] JOB optimizer_001 FINISHED
[03.04|14:45:48] Starting optimizer_001.postrun()
[03.04|14:45:48] optimizer_001.postrun() finished
[03.04|14:45:49] JOB optimizer_001 SUCCESSFUL
[03.04|14:45:49] PLAMS environment cleaned up successfully
[03.04|14:45:49] PLAMS run finished. Goodbye
[03.04|14:45:49] ParAMSResults
[03.04|14:45:49] Newly created parameter file/dir: step2_attempt2_training/results/optimization/optimizer_001/m3gnet
[03.04|14:45:49] Newly created parameter file/dir: step2_attempt2_training/results/optimization/optimizer_002/m3gnet
[03.04|14:45:49] Done!
[03.04|14:45:49] Deleting step2_attempt1_training
[03.04|14:45:49] ##########################
[03.04|14:45:49] ### Step 2 / Attempt 3 ###
[03.04|14:45:49] ##########################
[03.04|14:45:49] MD Steps: 31 (cumulative: 41)
[03.04|14:45:49] Current engine settings:
[03.04|14:45:49]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:45:49] Running step2_attempt3_simulation...
[03.04|14:46:10] Job step2_attempt3_simulation finished
[03.04|14:46:10] Deleting files that are no longer needed...
[03.04|14:46:10] Energy uncertainty for final frame of step2_attempt3_simulation: 0.1621 eV
[03.04|14:46:10] 0.0135 eV/atom
[03.04|14:46:10] Forces uncertainty for final frame of step2_attempt3_simulation: 0.2217 eV/angstrom
[03.04|14:46:11] Launching reference calculation
[03.04|14:46:19] Reference calculation finished!
[03.04|14:46:19] Checking success for step2_attempt3
[03.04|14:46:38] CheckEnergy: Checking energy for MDStep41, n_atoms = 12
[03.04|14:46:38] CheckEnergy: normalization coefficient = 12
[03.04|14:46:38] CheckEnergy: Actual Threshold
[03.04|14:46:38] CheckEnergy: dE/12 -0.0018 0.2000 OK!
[03.04|14:46:38] CheckEnergy: ddE/12 0.0009 0.0050 OK! (relative to step2_attempt2_simulation:MDStep41)
[03.04|14:46:38]
[03.04|14:46:38] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:46:38] CheckForces: ------------
[03.04|14:46:38] CheckForces: Reference job from step2_attempt3_reference_calc1
[03.04|14:46:38] CheckForces: Prediction job from final frame (MDStep41) of step2_attempt3_simulation
[03.04|14:46:38] CheckForces: ------------
[03.04|14:46:38] CheckForces: Histogram of forces
[03.04|14:46:38] CheckForces: eV/Ang Ref Pred
[03.04|14:46:38] CheckForces: -2 0 0
[03.04|14:46:38] CheckForces: -1 20 19
[03.04|14:46:38] CheckForces: 0 15 16
[03.04|14:46:38] CheckForces: 1 1 1
[03.04|14:46:38] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:46:38] CheckForces: All force components are within the acceptable error!
[03.04|14:46:38] CheckForces: Maximum deviation: 0.352 eV/angstrom
[03.04|14:46:38] CheckForces: Actual Threshold
[03.04|14:46:38] CheckForces: # > thr. 0 0 OK!
[03.04|14:46:38] CheckForces: MAE 0.105 0.30 OK!
[03.04|14:46:38] CheckForces: R^2 0.875 0.40 OK!
[03.04|14:46:38] CheckForces: --------------------
[03.04|14:46:38]
[03.04|14:46:38] Adding results from step2_attempt3_reference_calc1 to validation set
[03.04|14:46:38] Current # training set entries: 31
[03.04|14:46:38] Current # validation set entries: 11
[03.04|14:46:38] Storing data in step2_attempt3_reference_data
[03.04|14:46:39] Deleting step2_attempt2_reference_data
[03.04|14:46:39] Deleting step2_attempt3_reference_calc1
[03.04|14:46:39]
[03.04|14:46:39] Current (cumulative) timings:
[03.04|14:46:39] Time (s) Fraction
[03.04|14:46:39] Ref. calcs 140.68 0.114
[03.04|14:46:39] ML training 874.34 0.707
[03.04|14:46:39] Simulations 222.12 0.180
[03.04|14:46:39]
[03.04|14:46:39]
[03.04|14:46:39] Step 2 finished successfully!
[03.04|14:46:39]
[03.04|14:46:39] --- Begin summary ---
[03.04|14:46:39] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:46:39] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:46:39] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:46:39] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:46:39] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:46:39] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:46:39] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:46:39] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:46:39] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:46:39] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:46:39] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:46:39] --- End summary ---
[03.04|14:46:39]
[03.04|14:46:39] ##########################
[03.04|14:46:39] ### Step 3 / Attempt 1 ###
[03.04|14:46:39] ##########################
[03.04|14:46:39] MD Steps: 132 (cumulative: 173)
[03.04|14:46:39] Current engine settings:
[03.04|14:46:39]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step2_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:46:39] Running step3_attempt1_simulation...
[03.04|14:47:12] Job step3_attempt1_simulation finished
[03.04|14:47:12] Deleting files that are no longer needed...
[03.04|14:47:12] Deleting step1_attempt7_simulation
[03.04|14:47:12] Deleting step2_attempt1_simulation
[03.04|14:47:12] Deleting step2_attempt2_simulation
[03.04|14:47:12] Energy uncertainty for final frame of step3_attempt1_simulation: 0.0507 eV
[03.04|14:47:12] 0.0042 eV/atom
[03.04|14:47:12] Forces uncertainty for final frame of step3_attempt1_simulation: 0.5714 eV/angstrom
[03.04|14:47:13] Launching reference calculation
[03.04|14:47:20] Reference calculation finished!
[03.04|14:47:20] Checking success for step3_attempt1
[03.04|14:47:39] CheckEnergy: Checking energy for MDStep173, n_atoms = 12
[03.04|14:47:39] CheckEnergy: normalization coefficient = 12
[03.04|14:47:39] CheckEnergy: Actual Threshold
[03.04|14:47:39] CheckEnergy: dE/12 -0.1552 0.2000 OK!
[03.04|14:47:39] CheckEnergy: ddE/12 -0.1534 0.0050 Not OK! (relative to step2_attempt3_simulation:MDStep41)
[03.04|14:47:39]
[03.04|14:47:39] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:47:39] CheckForces: ------------
[03.04|14:47:39] CheckForces: Reference job from step3_attempt1_reference_calc1
[03.04|14:47:39] CheckForces: Prediction job from final frame (MDStep173) of step3_attempt1_simulation
[03.04|14:47:39] CheckForces: ------------
[03.04|14:47:39] CheckForces: Histogram of forces
[03.04|14:47:39] CheckForces: eV/Ang Ref Pred
[03.04|14:47:39] CheckForces: -3 0 0
[03.04|14:47:39] CheckForces: -2 3 0
[03.04|14:47:39] CheckForces: -1 12 19
[03.04|14:47:39] CheckForces: 0 18 17
[03.04|14:47:39] CheckForces: 1 3 0
[03.04|14:47:39] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:47:39] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:47:39] CheckForces: Ref Pred Delta Threshold
[03.04|14:47:39] CheckForces: 1.21 0.04 1.17 0.57
[03.04|14:47:39] CheckForces: 1.04 0.37 0.67 0.56
[03.04|14:47:39] CheckForces: 0.61 -0.08 0.69 0.53
[03.04|14:47:39] CheckForces: -0.97 0.14 1.10 0.55
[03.04|14:47:39] CheckForces: 0.59 -0.12 0.71 0.53
[03.04|14:47:39] CheckForces: 0.82 -0.19 1.00 0.54
[03.04|14:47:39] CheckForces: ... and 8 more.
[03.04|14:47:39] CheckForces: Maximum deviation: 1.756 eV/angstrom
[03.04|14:47:39] CheckForces: Actual Threshold
[03.04|14:47:39] CheckForces: # > thr. 14 0 Not OK!
[03.04|14:47:39] CheckForces: MAE 0.504 0.30 Not OK!
[03.04|14:47:39] CheckForces: R^2 0.043 0.40 Not OK!
[03.04|14:47:39] CheckForces: --------------------
[03.04|14:47:39]
[03.04|14:47:39] Adding results from step3_attempt1_reference_calc1 to training set
[03.04|14:47:39] Current # training set entries: 32
[03.04|14:47:39] Current # validation set entries: 11
[03.04|14:47:39] Storing data in step3_attempt1_reference_data
[03.04|14:47:39] Deleting step2_attempt3_reference_data
[03.04|14:47:39] Deleting step3_attempt1_reference_calc1
[03.04|14:47:39]
[03.04|14:47:39] Current (cumulative) timings:
[03.04|14:47:39] Time (s) Fraction
[03.04|14:47:39] Ref. calcs 147.60 0.116
[03.04|14:47:39] ML training 874.34 0.684
[03.04|14:47:39] Simulations 255.76 0.200
[03.04|14:47:39]
[03.04|14:47:39]
[03.04|14:47:39]
[03.04|14:47:39] --- Begin summary ---
[03.04|14:47:39] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:47:39] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:47:39] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:47:39] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:47:39] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:47:39] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:47:39] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:47:39] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:47:39] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:47:39] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:47:39] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:47:39] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|14:47:39] --- End summary ---
[03.04|14:47:39]
[03.04|14:47:39] Running more reference calculations....
[03.04|14:47:39] Running reference calculations on frames [26, 30] from step3_attempt1_simulation/ams.rkf
[03.04|14:47:39] Calculating 2 frames in total
[03.04|14:47:39] Running step3_attempt1_reference_calc2
[03.04|14:47:46] Running step3_attempt1_reference_calc3
[03.04|14:47:53] Reference calculations finished!
[03.04|14:47:53] Adding results from step3_attempt1_reference_calc2 to validation set
[03.04|14:47:53] Adding results from step3_attempt1_reference_calc3 to training set
[03.04|14:47:53] Current # training set entries: 33
[03.04|14:47:53] Current # validation set entries: 12
[03.04|14:47:53] Storing data in step3_attempt1_reference_data
[03.04|14:47:53] Deleting step3_attempt1_reference_calc2
[03.04|14:47:53] Deleting step3_attempt1_reference_calc3
[03.04|14:47:53] Launching reparametrization job: step3_attempt1_training
[03.04|14:47:57] JOB optimizer_001 STARTED
[03.04|14:47:57] JOB optimizer_002 STARTED
[03.04|14:47:57] Starting optimizer_001.prerun()
[03.04|14:47:57] Starting optimizer_002.prerun()
[03.04|14:47:57] optimizer_001.prerun() finished
[03.04|14:47:57] optimizer_002.prerun() finished
[03.04|14:47:57] JOB optimizer_001 RUNNING
[03.04|14:47:57] Executing optimizer_001.run
[03.04|14:47:57] JOB optimizer_002 RUNNING
[03.04|14:47:57] Executing optimizer_002.run
[03.04|14:47:57] Waiting for job optimizer_001 to finish
[03.04|14:49:05] training_set Optimizer: 001 Epoch: 0 Loss: 0.008390
[03.04|14:49:05] training_set Optimizer: 002 Epoch: 0 Loss: 0.006780
[03.04|14:49:05] validation_set Optimizer: 001 Epoch: 0 Loss: 0.044066
[03.04|14:49:05] validation_set Optimizer: 002 Epoch: 0 Loss: 0.031543
[03.04|14:49:09] training_set Optimizer: 001 Epoch: 10 Loss: 0.004019
[03.04|14:49:09] validation_set Optimizer: 001 Epoch: 10 Loss: 0.024502
[03.04|14:49:09] training_set Optimizer: 002 Epoch: 10 Loss: 0.004291
[03.04|14:49:09] validation_set Optimizer: 002 Epoch: 10 Loss: 0.024874
[03.04|14:49:13] training_set Optimizer: 001 Epoch: 20 Loss: 0.004229
[03.04|14:49:13] validation_set Optimizer: 001 Epoch: 20 Loss: 0.021847
[03.04|14:49:13] training_set Optimizer: 002 Epoch: 20 Loss: 0.003939
[03.04|14:49:13] validation_set Optimizer: 002 Epoch: 20 Loss: 0.035504
[03.04|14:49:17] training_set Optimizer: 001 Epoch: 30 Loss: 0.003606
[03.04|14:49:17] validation_set Optimizer: 001 Epoch: 30 Loss: 0.021881
[03.04|14:49:17] training_set Optimizer: 002 Epoch: 30 Loss: 0.003814
[03.04|14:49:17] validation_set Optimizer: 002 Epoch: 30 Loss: 0.027407
[03.04|14:49:21] training_set Optimizer: 002 Epoch: 40 Loss: 0.003443
[03.04|14:49:21] training_set Optimizer: 001 Epoch: 40 Loss: 0.004307
[03.04|14:49:21] validation_set Optimizer: 001 Epoch: 40 Loss: 0.046528
[03.04|14:49:21] validation_set Optimizer: 002 Epoch: 40 Loss: 0.024092
[03.04|14:49:25] training_set Optimizer: 002 Epoch: 50 Loss: 0.004077
[03.04|14:49:25] training_set Optimizer: 001 Epoch: 50 Loss: 0.004238
[03.04|14:49:25] validation_set Optimizer: 002 Epoch: 50 Loss: 0.026959
[03.04|14:49:25] validation_set Optimizer: 001 Epoch: 50 Loss: 0.029735
[03.04|14:49:29] training_set Optimizer: 002 Epoch: 60 Loss: 0.003511
[03.04|14:49:29] training_set Optimizer: 001 Epoch: 60 Loss: 0.003747
[03.04|14:49:29] validation_set Optimizer: 002 Epoch: 60 Loss: 0.031545
[03.04|14:49:29] validation_set Optimizer: 001 Epoch: 60 Loss: 0.023856
[03.04|14:49:33] training_set Optimizer: 001 Epoch: 70 Loss: 0.003242
[03.04|14:49:33] validation_set Optimizer: 001 Epoch: 70 Loss: 0.017516
[03.04|14:49:34] training_set Optimizer: 002 Epoch: 70 Loss: 0.003163
[03.04|14:49:34] validation_set Optimizer: 002 Epoch: 70 Loss: 0.023826
[03.04|14:49:38] training_set Optimizer: 001 Epoch: 80 Loss: 0.003159
[03.04|14:49:38] validation_set Optimizer: 001 Epoch: 80 Loss: 0.026173
[03.04|14:49:38] training_set Optimizer: 002 Epoch: 80 Loss: 0.003478
[03.04|14:49:38] validation_set Optimizer: 002 Epoch: 80 Loss: 0.026731
[03.04|14:49:42] training_set Optimizer: 001 Epoch: 90 Loss: 0.003380
[03.04|14:49:42] training_set Optimizer: 002 Epoch: 90 Loss: 0.003223
[03.04|14:49:42] validation_set Optimizer: 001 Epoch: 90 Loss: 0.023021
[03.04|14:49:42] validation_set Optimizer: 002 Epoch: 90 Loss: 0.025107
[03.04|14:49:46] training_set Optimizer: 002 Epoch: 100 Loss: 0.003094
[03.04|14:49:46] validation_set Optimizer: 002 Epoch: 100 Loss: 0.034616
[03.04|14:49:46] training_set Optimizer: 001 Epoch: 100 Loss: 0.002791
[03.04|14:49:46] validation_set Optimizer: 001 Epoch: 100 Loss: 0.027896
[03.04|14:49:50] training_set Optimizer: 001 Epoch: 110 Loss: 0.002948
[03.04|14:49:50] validation_set Optimizer: 001 Epoch: 110 Loss: 0.026005
[03.04|14:49:50] training_set Optimizer: 002 Epoch: 110 Loss: 0.003098
[03.04|14:49:50] validation_set Optimizer: 002 Epoch: 110 Loss: 0.029158
[03.04|14:49:56] Execution of optimizer_001.run finished with returncode 0
[03.04|14:49:56] Execution of optimizer_002.run finished with returncode 0
[03.04|14:49:56] JOB optimizer_002 FINISHED
[03.04|14:49:56] Starting optimizer_002.postrun()
[03.04|14:49:56] optimizer_002.postrun() finished
[03.04|14:49:56] JOB optimizer_001 FINISHED
[03.04|14:49:56] Starting optimizer_001.postrun()
[03.04|14:49:56] optimizer_001.postrun() finished
[03.04|14:49:56] JOB optimizer_002 SUCCESSFUL
[03.04|14:49:56] JOB optimizer_001 SUCCESSFUL
[03.04|14:49:56] PLAMS environment cleaned up successfully
[03.04|14:49:56] PLAMS run finished. Goodbye
[03.04|14:49:57] ParAMSResults
[03.04|14:49:57] Newly created parameter file/dir: step3_attempt1_training/results/optimization/optimizer_001/m3gnet
[03.04|14:49:57] Newly created parameter file/dir: step3_attempt1_training/results/optimization/optimizer_002/m3gnet
[03.04|14:49:57] Done!
[03.04|14:49:57] Deleting step2_attempt2_training
[03.04|14:49:57] ##########################
[03.04|14:49:57] ### Step 3 / Attempt 2 ###
[03.04|14:49:57] ##########################
[03.04|14:49:57] MD Steps: 132 (cumulative: 173)
[03.04|14:49:57] Current engine settings:
[03.04|14:49:57]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt1_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt1_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:49:57] Running step3_attempt2_simulation...
[03.04|14:50:31] Job step3_attempt2_simulation finished
[03.04|14:50:31] Deleting files that are no longer needed...
[03.04|14:50:31] Energy uncertainty for final frame of step3_attempt2_simulation: 0.0391 eV
[03.04|14:50:31] 0.0033 eV/atom
[03.04|14:50:31] Forces uncertainty for final frame of step3_attempt2_simulation: 0.6386 eV/angstrom
[03.04|14:50:31] Launching reference calculation
[03.04|14:50:39] Reference calculation finished!
[03.04|14:50:39] Checking success for step3_attempt2
[03.04|14:50:59] CheckEnergy: Checking energy for MDStep173, n_atoms = 12
[03.04|14:50:59] CheckEnergy: normalization coefficient = 12
[03.04|14:50:59] CheckEnergy: Actual Threshold
[03.04|14:50:59] CheckEnergy: dE/12 -0.0147 0.2000 OK!
[03.04|14:50:59] CheckEnergy: ddE/12 0.0656 0.0050 Not OK! (relative to step3_attempt1_simulation:MDStep173)
[03.04|14:50:59]
[03.04|14:50:59] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:50:59] CheckForces: ------------
[03.04|14:50:59] CheckForces: Reference job from step3_attempt2_reference_calc1
[03.04|14:50:59] CheckForces: Prediction job from final frame (MDStep173) of step3_attempt2_simulation
[03.04|14:50:59] CheckForces: ------------
[03.04|14:50:59] CheckForces: Histogram of forces
[03.04|14:50:59] CheckForces: eV/Ang Ref Pred
[03.04|14:50:59] CheckForces: -2 0 0
[03.04|14:50:59] CheckForces: -1 18 18
[03.04|14:50:59] CheckForces: 0 17 18
[03.04|14:50:59] CheckForces: 1 1 0
[03.04|14:50:59] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:50:59] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:50:59] CheckForces: Ref Pred Delta Threshold
[03.04|14:50:59] CheckForces: -0.35 0.33 0.68 0.52
[03.04|14:50:59] CheckForces: Maximum deviation: 0.677 eV/angstrom
[03.04|14:50:59] CheckForces: Actual Threshold
[03.04|14:50:59] CheckForces: # > thr. 1 0 Not OK!
[03.04|14:50:59] CheckForces: MAE 0.183 0.30 OK!
[03.04|14:50:59] CheckForces: R^2 0.588 0.40 OK!
[03.04|14:50:59] CheckForces: --------------------
[03.04|14:50:59]
[03.04|14:50:59] Adding results from step3_attempt2_reference_calc1 to training set
[03.04|14:50:59] Current # training set entries: 34
[03.04|14:50:59] Current # validation set entries: 12
[03.04|14:50:59] Storing data in step3_attempt2_reference_data
[03.04|14:50:59] Deleting step3_attempt1_reference_data
[03.04|14:50:59] Deleting step3_attempt2_reference_calc1
[03.04|14:50:59]
[03.04|14:50:59] Current (cumulative) timings:
[03.04|14:50:59] Time (s) Fraction
[03.04|14:50:59] Ref. calcs 168.90 0.116
[03.04|14:50:59] ML training 998.08 0.685
[03.04|14:50:59] Simulations 289.94 0.199
[03.04|14:50:59]
[03.04|14:50:59]
[03.04|14:50:59]
[03.04|14:50:59] --- Begin summary ---
[03.04|14:50:59] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:50:59] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:50:59] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:50:59] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:50:59] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:50:59] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:50:59] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:50:59] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:50:59] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:50:59] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:50:59] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:50:59] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|14:50:59] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|14:50:59] --- End summary ---
[03.04|14:50:59]
[03.04|14:50:59] Running more reference calculations....
[03.04|14:50:59] Running reference calculations on frames [26, 30] from step3_attempt2_simulation/ams.rkf
[03.04|14:50:59] Calculating 2 frames in total
[03.04|14:50:59] Running step3_attempt2_reference_calc2
[03.04|14:51:07] Running step3_attempt2_reference_calc3
[03.04|14:51:15] Reference calculations finished!
[03.04|14:51:15] Adding results from step3_attempt2_reference_calc2 to validation set
[03.04|14:51:15] Adding results from step3_attempt2_reference_calc3 to training set
[03.04|14:51:15] Current # training set entries: 35
[03.04|14:51:15] Current # validation set entries: 13
[03.04|14:51:15] Storing data in step3_attempt2_reference_data
[03.04|14:51:15] Deleting step3_attempt2_reference_calc2
[03.04|14:51:15] Deleting step3_attempt2_reference_calc3
[03.04|14:51:15] Launching reparametrization job: step3_attempt2_training
[03.04|14:51:19] JOB optimizer_001 STARTED
[03.04|14:51:19] JOB optimizer_002 STARTED
[03.04|14:51:19] Starting optimizer_001.prerun()
[03.04|14:51:19] optimizer_001.prerun() finished
[03.04|14:51:19] Starting optimizer_002.prerun()
[03.04|14:51:19] optimizer_002.prerun() finished
[03.04|14:51:19] JOB optimizer_002 RUNNING
[03.04|14:51:19] JOB optimizer_001 RUNNING
[03.04|14:51:19] Executing optimizer_002.run
[03.04|14:51:19] Executing optimizer_001.run
[03.04|14:51:19] Waiting for job optimizer_001 to finish
[03.04|14:52:08] training_set Optimizer: 001 Epoch: 0 Loss: 0.005299
[03.04|14:52:08] validation_set Optimizer: 001 Epoch: 0 Loss: 0.017588
[03.04|14:52:08] training_set Optimizer: 002 Epoch: 0 Loss: 0.004932
[03.04|14:52:08] validation_set Optimizer: 002 Epoch: 0 Loss: 0.041514
[03.04|14:52:12] training_set Optimizer: 001 Epoch: 10 Loss: 0.003138
[03.04|14:52:12] validation_set Optimizer: 001 Epoch: 10 Loss: 0.031367
[03.04|14:52:12] training_set Optimizer: 002 Epoch: 10 Loss: 0.002877
[03.04|14:52:12] validation_set Optimizer: 002 Epoch: 10 Loss: 0.019857
[03.04|14:52:16] training_set Optimizer: 001 Epoch: 20 Loss: 0.003078
[03.04|14:52:16] validation_set Optimizer: 001 Epoch: 20 Loss: 0.032717
[03.04|14:52:16] training_set Optimizer: 002 Epoch: 20 Loss: 0.002773
[03.04|14:52:16] validation_set Optimizer: 002 Epoch: 20 Loss: 0.024241
[03.04|14:52:20] training_set Optimizer: 001 Epoch: 30 Loss: 0.003193
[03.04|14:52:20] validation_set Optimizer: 001 Epoch: 30 Loss: 0.019269
[03.04|14:52:21] training_set Optimizer: 002 Epoch: 30 Loss: 0.003065
[03.04|14:52:21] validation_set Optimizer: 002 Epoch: 30 Loss: 0.026958
[03.04|14:52:24] training_set Optimizer: 001 Epoch: 40 Loss: 0.002718
[03.04|14:52:24] validation_set Optimizer: 001 Epoch: 40 Loss: 0.015633
[03.04|14:52:25] training_set Optimizer: 002 Epoch: 40 Loss: 0.002650
[03.04|14:52:25] validation_set Optimizer: 002 Epoch: 40 Loss: 0.017272
[03.04|14:52:28] training_set Optimizer: 001 Epoch: 50 Loss: 0.002697
[03.04|14:52:28] validation_set Optimizer: 001 Epoch: 50 Loss: 0.014152
[03.04|14:52:29] training_set Optimizer: 002 Epoch: 50 Loss: 0.002473
[03.04|14:52:29] validation_set Optimizer: 002 Epoch: 50 Loss: 0.021200
[03.04|14:52:32] training_set Optimizer: 001 Epoch: 60 Loss: 0.002660
[03.04|14:52:32] validation_set Optimizer: 001 Epoch: 60 Loss: 0.018918
[03.04|14:52:33] training_set Optimizer: 002 Epoch: 60 Loss: 0.002505
[03.04|14:52:33] validation_set Optimizer: 002 Epoch: 60 Loss: 0.020520
[03.04|14:52:36] training_set Optimizer: 001 Epoch: 70 Loss: 0.002786
[03.04|14:52:36] validation_set Optimizer: 001 Epoch: 70 Loss: 0.013752
[03.04|14:52:37] training_set Optimizer: 002 Epoch: 70 Loss: 0.002628
[03.04|14:52:37] validation_set Optimizer: 002 Epoch: 70 Loss: 0.028672
[03.04|14:52:40] training_set Optimizer: 001 Epoch: 80 Loss: 0.002594
[03.04|14:52:40] validation_set Optimizer: 001 Epoch: 80 Loss: 0.017108
[03.04|14:52:41] training_set Optimizer: 002 Epoch: 80 Loss: 0.002413
[03.04|14:52:41] validation_set Optimizer: 002 Epoch: 80 Loss: 0.020665
[03.04|14:52:45] training_set Optimizer: 001 Epoch: 90 Loss: 0.003041
[03.04|14:52:45] validation_set Optimizer: 001 Epoch: 90 Loss: 0.019132
[03.04|14:52:46] training_set Optimizer: 002 Epoch: 90 Loss: 0.002253
[03.04|14:52:46] validation_set Optimizer: 002 Epoch: 90 Loss: 0.014744
[03.04|14:52:49] training_set Optimizer: 001 Epoch: 100 Loss: 0.002707
[03.04|14:52:49] validation_set Optimizer: 001 Epoch: 100 Loss: 0.030299
[03.04|14:52:50] training_set Optimizer: 002 Epoch: 100 Loss: 0.002532
[03.04|14:52:50] validation_set Optimizer: 002 Epoch: 100 Loss: 0.017712
[03.04|14:52:53] training_set Optimizer: 001 Epoch: 110 Loss: 0.002423
[03.04|14:52:53] validation_set Optimizer: 001 Epoch: 110 Loss: 0.040136
[03.04|14:52:54] training_set Optimizer: 002 Epoch: 110 Loss: 0.002205
[03.04|14:52:54] validation_set Optimizer: 002 Epoch: 110 Loss: 0.014803
[03.04|14:52:58] Execution of optimizer_001.run finished with returncode 0
[03.04|14:52:58] JOB optimizer_001 FINISHED
[03.04|14:52:58] Starting optimizer_001.postrun()
[03.04|14:52:58] optimizer_001.postrun() finished
[03.04|14:52:58] JOB optimizer_001 SUCCESSFUL
[03.04|14:52:58] Waiting for job optimizer_002 to finish
[03.04|14:52:58] Execution of optimizer_002.run finished with returncode 0
[03.04|14:52:59] JOB optimizer_002 FINISHED
[03.04|14:52:59] Starting optimizer_002.postrun()
[03.04|14:52:59] optimizer_002.postrun() finished
[03.04|14:52:59] JOB optimizer_002 SUCCESSFUL
[03.04|14:52:59] PLAMS environment cleaned up successfully
[03.04|14:52:59] PLAMS run finished. Goodbye
[03.04|14:52:59] ParAMSResults
[03.04|14:52:59] Newly created parameter file/dir: step3_attempt2_training/results/optimization/optimizer_001/m3gnet
[03.04|14:52:59] Newly created parameter file/dir: step3_attempt2_training/results/optimization/optimizer_002/m3gnet
[03.04|14:52:59] Done!
[03.04|14:52:59] Deleting step3_attempt1_training
[03.04|14:52:59] ##########################
[03.04|14:52:59] ### Step 3 / Attempt 3 ###
[03.04|14:52:59] ##########################
[03.04|14:52:59] MD Steps: 132 (cumulative: 173)
[03.04|14:52:59] Current engine settings:
[03.04|14:52:59]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:52:59] Running step3_attempt3_simulation...
[03.04|14:53:27] Job step3_attempt3_simulation finished
[03.04|14:53:27] Deleting files that are no longer needed...
[03.04|14:53:27] Energy uncertainty for final frame of step3_attempt3_simulation: 0.0059 eV
[03.04|14:53:27] 0.0005 eV/atom
[03.04|14:53:27] Forces uncertainty for final frame of step3_attempt3_simulation: 0.1751 eV/angstrom
[03.04|14:53:28] Launching reference calculation
[03.04|14:53:37] Reference calculation finished!
[03.04|14:53:37] Checking success for step3_attempt3
[03.04|14:53:57] CheckEnergy: Checking energy for MDStep173, n_atoms = 12
[03.04|14:53:57] CheckEnergy: normalization coefficient = 12
[03.04|14:53:57] CheckEnergy: Actual Threshold
[03.04|14:53:57] CheckEnergy: dE/12 0.0074 0.2000 OK!
[03.04|14:53:57] CheckEnergy: ddE/12 0.0060 0.0050 Not OK! (relative to step3_attempt2_simulation:MDStep173)
[03.04|14:53:57]
[03.04|14:53:57] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:53:57] CheckForces: ------------
[03.04|14:53:57] CheckForces: Reference job from step3_attempt3_reference_calc1
[03.04|14:53:57] CheckForces: Prediction job from final frame (MDStep173) of step3_attempt3_simulation
[03.04|14:53:57] CheckForces: ------------
[03.04|14:53:57] CheckForces: Histogram of forces
[03.04|14:53:57] CheckForces: eV/Ang Ref Pred
[03.04|14:53:57] CheckForces: -2 0 0
[03.04|14:53:57] CheckForces: -1 18 21
[03.04|14:53:57] CheckForces: 0 18 15
[03.04|14:53:57] CheckForces: 1 0 0
[03.04|14:53:57] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:53:57] CheckForces: All force components are within the acceptable error!
[03.04|14:53:57] CheckForces: Maximum deviation: 0.426 eV/angstrom
[03.04|14:53:57] CheckForces: Actual Threshold
[03.04|14:53:57] CheckForces: # > thr. 0 0 OK!
[03.04|14:53:57] CheckForces: MAE 0.099 0.30 OK!
[03.04|14:53:57] CheckForces: R^2 0.750 0.40 OK!
[03.04|14:53:57] CheckForces: --------------------
[03.04|14:53:57]
[03.04|14:53:57] Adding results from step3_attempt3_reference_calc1 to training set
[03.04|14:53:57] Current # training set entries: 36
[03.04|14:53:57] Current # validation set entries: 13
[03.04|14:53:57] Storing data in step3_attempt3_reference_data
[03.04|14:53:57] Deleting step3_attempt2_reference_data
[03.04|14:53:57] Deleting step3_attempt3_reference_calc1
[03.04|14:53:57]
[03.04|14:53:57] Current (cumulative) timings:
[03.04|14:53:57] Time (s) Fraction
[03.04|14:53:57] Ref. calcs 194.30 0.120
[03.04|14:53:57] ML training 1102.09 0.683
[03.04|14:53:57] Simulations 317.99 0.197
[03.04|14:53:57]
[03.04|14:53:57]
[03.04|14:53:57]
[03.04|14:53:57] --- Begin summary ---
[03.04|14:53:57] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:53:57] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:53:57] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:53:57] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:53:57] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:53:57] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:53:57] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:53:57] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:53:57] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:53:57] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:53:57] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:53:57] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|14:53:57] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|14:53:57] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|14:53:57] --- End summary ---
[03.04|14:53:57]
[03.04|14:53:57] Running more reference calculations....
[03.04|14:53:57] Running reference calculations on frames [26, 30] from step3_attempt3_simulation/ams.rkf
[03.04|14:53:57] Calculating 2 frames in total
[03.04|14:53:57] Running step3_attempt3_reference_calc2
[03.04|14:54:05] Running step3_attempt3_reference_calc3
[03.04|14:54:12] Reference calculations finished!
[03.04|14:54:13] Adding results from step3_attempt3_reference_calc2 to validation set
[03.04|14:54:13] Adding results from step3_attempt3_reference_calc3 to training set
[03.04|14:54:13] Current # training set entries: 37
[03.04|14:54:13] Current # validation set entries: 14
[03.04|14:54:13] Storing data in step3_attempt3_reference_data
[03.04|14:54:13] Deleting step3_attempt3_reference_calc2
[03.04|14:54:13] Deleting step3_attempt3_reference_calc3
[03.04|14:54:13] Launching reparametrization job: step3_attempt3_training
[03.04|14:54:17] JOB optimizer_001 STARTED
[03.04|14:54:17] JOB optimizer_002 STARTED
[03.04|14:54:17] Starting optimizer_001.prerun()
[03.04|14:54:17] Starting optimizer_002.prerun()
[03.04|14:54:17] optimizer_001.prerun() finished
[03.04|14:54:17] optimizer_002.prerun() finished
[03.04|14:54:17] JOB optimizer_002 RUNNING
[03.04|14:54:17] JOB optimizer_001 RUNNING
[03.04|14:54:17] Executing optimizer_002.run
[03.04|14:54:17] Executing optimizer_001.run
[03.04|14:54:17] Waiting for job optimizer_001 to finish
[03.04|14:55:25] training_set Optimizer: 001 Epoch: 0 Loss: 0.006816
[03.04|14:55:25] validation_set Optimizer: 001 Epoch: 0 Loss: 0.076447
[03.04|14:55:25] training_set Optimizer: 002 Epoch: 0 Loss: 0.004328
[03.04|14:55:25] validation_set Optimizer: 002 Epoch: 0 Loss: 0.040426
[03.04|14:55:29] training_set Optimizer: 001 Epoch: 10 Loss: 0.002279
[03.04|14:55:29] validation_set Optimizer: 001 Epoch: 10 Loss: 0.017171
[03.04|14:55:30] training_set Optimizer: 002 Epoch: 10 Loss: 0.002133
[03.04|14:55:30] validation_set Optimizer: 002 Epoch: 10 Loss: 0.019857
[03.04|14:55:34] training_set Optimizer: 002 Epoch: 20 Loss: 0.002327
[03.04|14:55:34] training_set Optimizer: 001 Epoch: 20 Loss: 0.002413
[03.04|14:55:34] validation_set Optimizer: 002 Epoch: 20 Loss: 0.016563
[03.04|14:55:34] validation_set Optimizer: 001 Epoch: 20 Loss: 0.015092
[03.04|14:55:39] training_set Optimizer: 002 Epoch: 30 Loss: 0.002169
[03.04|14:55:39] training_set Optimizer: 001 Epoch: 30 Loss: 0.002176
[03.04|14:55:39] validation_set Optimizer: 002 Epoch: 30 Loss: 0.014696
[03.04|14:55:39] validation_set Optimizer: 001 Epoch: 30 Loss: 0.013638
[03.04|14:55:43] training_set Optimizer: 002 Epoch: 40 Loss: 0.001902
[03.04|14:55:43] training_set Optimizer: 001 Epoch: 40 Loss: 0.002226
[03.04|14:55:43] validation_set Optimizer: 002 Epoch: 40 Loss: 0.020813
[03.04|14:55:43] validation_set Optimizer: 001 Epoch: 40 Loss: 0.012491
[03.04|14:55:48] training_set Optimizer: 001 Epoch: 50 Loss: 0.002444
[03.04|14:55:48] validation_set Optimizer: 001 Epoch: 50 Loss: 0.037438
[03.04|14:55:48] training_set Optimizer: 002 Epoch: 50 Loss: 0.001897
[03.04|14:55:48] validation_set Optimizer: 002 Epoch: 50 Loss: 0.015000
[03.04|14:55:52] training_set Optimizer: 001 Epoch: 60 Loss: 0.002154
[03.04|14:55:52] validation_set Optimizer: 001 Epoch: 60 Loss: 0.012985
[03.04|14:55:53] training_set Optimizer: 002 Epoch: 60 Loss: 0.002048
[03.04|14:55:53] validation_set Optimizer: 002 Epoch: 60 Loss: 0.028350
[03.04|14:55:57] training_set Optimizer: 001 Epoch: 70 Loss: 0.002192
[03.04|14:55:57] validation_set Optimizer: 001 Epoch: 70 Loss: 0.012855
[03.04|14:55:57] training_set Optimizer: 002 Epoch: 70 Loss: 0.001964
[03.04|14:55:57] validation_set Optimizer: 002 Epoch: 70 Loss: 0.023354
[03.04|14:56:02] training_set Optimizer: 001 Epoch: 80 Loss: 0.002124
[03.04|14:56:02] training_set Optimizer: 002 Epoch: 80 Loss: 0.001999
[03.04|14:56:02] validation_set Optimizer: 001 Epoch: 80 Loss: 0.020708
[03.04|14:56:02] validation_set Optimizer: 002 Epoch: 80 Loss: 0.019702
[03.04|14:56:06] training_set Optimizer: 002 Epoch: 90 Loss: 0.001961
[03.04|14:56:06] training_set Optimizer: 001 Epoch: 90 Loss: 0.001988
[03.04|14:56:06] validation_set Optimizer: 002 Epoch: 90 Loss: 0.016491
[03.04|14:56:06] validation_set Optimizer: 001 Epoch: 90 Loss: 0.020510
[03.04|14:56:11] training_set Optimizer: 001 Epoch: 100 Loss: 0.001950
[03.04|14:56:11] training_set Optimizer: 002 Epoch: 100 Loss: 0.001999
[03.04|14:56:11] validation_set Optimizer: 001 Epoch: 100 Loss: 0.016678
[03.04|14:56:11] validation_set Optimizer: 002 Epoch: 100 Loss: 0.021335
[03.04|14:56:15] training_set Optimizer: 001 Epoch: 110 Loss: 0.002104
[03.04|14:56:15] validation_set Optimizer: 001 Epoch: 110 Loss: 0.014465
[03.04|14:56:16] training_set Optimizer: 002 Epoch: 110 Loss: 0.002244
[03.04|14:56:16] validation_set Optimizer: 002 Epoch: 110 Loss: 0.028551
[03.04|14:56:22] Execution of optimizer_002.run finished with returncode 0
[03.04|14:56:22] Execution of optimizer_001.run finished with returncode 0
[03.04|14:56:22] JOB optimizer_001 FINISHED
[03.04|14:56:22] Starting optimizer_001.postrun()
[03.04|14:56:22] optimizer_001.postrun() finished
[03.04|14:56:22] JOB optimizer_002 FINISHED
[03.04|14:56:22] Starting optimizer_002.postrun()
[03.04|14:56:22] optimizer_002.postrun() finished
[03.04|14:56:22] JOB optimizer_001 SUCCESSFUL
[03.04|14:56:22] JOB optimizer_002 SUCCESSFUL
[03.04|14:56:22] PLAMS environment cleaned up successfully
[03.04|14:56:22] PLAMS run finished. Goodbye
[03.04|14:56:23] ParAMSResults
[03.04|14:56:23] Newly created parameter file/dir: step3_attempt3_training/results/optimization/optimizer_001/m3gnet
[03.04|14:56:23] Newly created parameter file/dir: step3_attempt3_training/results/optimization/optimizer_002/m3gnet
[03.04|14:56:23] Done!
[03.04|14:56:23] Deleting step3_attempt2_training
[03.04|14:56:23] ##########################
[03.04|14:56:23] ### Step 3 / Attempt 4 ###
[03.04|14:56:23] ##########################
[03.04|14:56:23] MD Steps: 132 (cumulative: 173)
[03.04|14:56:23] Current engine settings:
[03.04|14:56:23]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt3_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt3_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:56:23] Running step3_attempt4_simulation...
[03.04|14:56:51] Job step3_attempt4_simulation finished
[03.04|14:56:51] Deleting files that are no longer needed...
[03.04|14:56:51] Energy uncertainty for final frame of step3_attempt4_simulation: 0.0333 eV
[03.04|14:56:51] 0.0028 eV/atom
[03.04|14:56:51] Forces uncertainty for final frame of step3_attempt4_simulation: 0.1976 eV/angstrom
[03.04|14:56:51] Launching reference calculation
[03.04|14:56:59] Reference calculation finished!
[03.04|14:56:59] Checking success for step3_attempt4
[03.04|14:57:18] CheckEnergy: Checking energy for MDStep173, n_atoms = 12
[03.04|14:57:18] CheckEnergy: normalization coefficient = 12
[03.04|14:57:18] CheckEnergy: Actual Threshold
[03.04|14:57:18] CheckEnergy: dE/12 0.0019 0.2000 OK!
[03.04|14:57:18] CheckEnergy: ddE/12 0.0002 0.0050 OK! (relative to step3_attempt3_simulation:MDStep173)
[03.04|14:57:18]
[03.04|14:57:18] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:57:18] CheckForces: ------------
[03.04|14:57:18] CheckForces: Reference job from step3_attempt4_reference_calc1
[03.04|14:57:18] CheckForces: Prediction job from final frame (MDStep173) of step3_attempt4_simulation
[03.04|14:57:18] CheckForces: ------------
[03.04|14:57:18] CheckForces: Histogram of forces
[03.04|14:57:18] CheckForces: eV/Ang Ref Pred
[03.04|14:57:18] CheckForces: -2 0 0
[03.04|14:57:18] CheckForces: -1 18 21
[03.04|14:57:18] CheckForces: 0 18 15
[03.04|14:57:18] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:57:18] CheckForces: All force components are within the acceptable error!
[03.04|14:57:18] CheckForces: Maximum deviation: 0.215 eV/angstrom
[03.04|14:57:18] CheckForces: Actual Threshold
[03.04|14:57:18] CheckForces: # > thr. 0 0 OK!
[03.04|14:57:18] CheckForces: MAE 0.060 0.30 OK!
[03.04|14:57:18] CheckForces: R^2 0.909 0.40 OK!
[03.04|14:57:18] CheckForces: --------------------
[03.04|14:57:18]
[03.04|14:57:18] Adding results from step3_attempt4_reference_calc1 to training set
[03.04|14:57:18] Current # training set entries: 38
[03.04|14:57:18] Current # validation set entries: 14
[03.04|14:57:18] Storing data in step3_attempt4_reference_data
[03.04|14:57:18] Deleting step3_attempt3_reference_data
[03.04|14:57:18] Deleting step3_attempt4_reference_calc1
[03.04|14:57:18]
[03.04|14:57:18] Current (cumulative) timings:
[03.04|14:57:18] Time (s) Fraction
[03.04|14:57:18] Ref. calcs 216.79 0.121
[03.04|14:57:18] ML training 1232.16 0.686
[03.04|14:57:18] Simulations 346.24 0.193
[03.04|14:57:18]
[03.04|14:57:18]
[03.04|14:57:18] Step 3 finished successfully!
[03.04|14:57:18]
[03.04|14:57:18] --- Begin summary ---
[03.04|14:57:18] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:57:18] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:57:18] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:57:18] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:57:18] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:57:18] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:57:18] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:57:18] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:57:18] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:57:18] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:57:18] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:57:18] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|14:57:18] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|14:57:18] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|14:57:18] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|14:57:18] --- End summary ---
[03.04|14:57:18]
[03.04|14:57:18] ##########################
[03.04|14:57:18] ### Step 4 / Attempt 1 ###
[03.04|14:57:18] ##########################
[03.04|14:57:18] MD Steps: 547 (cumulative: 720)
[03.04|14:57:18] Current engine settings:
[03.04|14:57:18]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt3_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step3_attempt3_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|14:57:18] Running step4_attempt1_simulation...
[03.04|14:58:01] Job step4_attempt1_simulation finished
[03.04|14:58:01] Deleting files that are no longer needed...
[03.04|14:58:01] Deleting step2_attempt3_simulation
[03.04|14:58:01] Deleting step3_attempt1_simulation
[03.04|14:58:01] Deleting step3_attempt2_simulation
[03.04|14:58:01] Deleting step3_attempt3_simulation
[03.04|14:58:01] Energy uncertainty for final frame of step4_attempt1_simulation: 0.3782 eV
[03.04|14:58:01] 0.0315 eV/atom
[03.04|14:58:01] Forces uncertainty for final frame of step4_attempt1_simulation: 0.5658 eV/angstrom
[03.04|14:58:02] Launching reference calculation
[03.04|14:58:09] Reference calculation finished!
[03.04|14:58:09] Checking success for step4_attempt1
[03.04|14:58:30] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|14:58:30] CheckEnergy: normalization coefficient = 12
[03.04|14:58:30] CheckEnergy: Actual Threshold
[03.04|14:58:30] CheckEnergy: dE/12 -0.0387 0.2000 OK!
[03.04|14:58:30] CheckEnergy: ddE/12 -0.0406 0.0050 Not OK! (relative to step3_attempt4_simulation:MDStep173)
[03.04|14:58:30]
[03.04|14:58:30] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|14:58:30] CheckForces: ------------
[03.04|14:58:30] CheckForces: Reference job from step4_attempt1_reference_calc1
[03.04|14:58:30] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt1_simulation
[03.04|14:58:30] CheckForces: ------------
[03.04|14:58:30] CheckForces: Histogram of forces
[03.04|14:58:30] CheckForces: eV/Ang Ref Pred
[03.04|14:58:30] CheckForces: -2 0 0
[03.04|14:58:30] CheckForces: -1 18 18
[03.04|14:58:30] CheckForces: 0 18 18
[03.04|14:58:30] CheckForces: 1 0 0
[03.04|14:58:30] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|14:58:30] CheckForces: Force components with an error exceeding the threshold:
[03.04|14:58:30] CheckForces: Ref Pred Delta Threshold
[03.04|14:58:30] CheckForces: 0.15 0.68 0.53 0.51
[03.04|14:58:30] CheckForces: 0.86 0.07 0.79 0.55
[03.04|14:58:30] CheckForces: -0.79 -0.15 0.65 0.54
[03.04|14:58:30] CheckForces: -0.65 -0.10 0.55 0.53
[03.04|14:58:30] CheckForces: 0.17 -0.63 0.80 0.51
[03.04|14:58:30] CheckForces: Maximum deviation: 0.801 eV/angstrom
[03.04|14:58:30] CheckForces: Actual Threshold
[03.04|14:58:30] CheckForces: # > thr. 5 0 Not OK!
[03.04|14:58:30] CheckForces: MAE 0.328 0.30 Not OK!
[03.04|14:58:30] CheckForces: R^2 0.235 0.40 Not OK!
[03.04|14:58:30] CheckForces: --------------------
[03.04|14:58:30]
[03.04|14:58:30] Adding results from step4_attempt1_reference_calc1 to training set
[03.04|14:58:30] Current # training set entries: 39
[03.04|14:58:30] Current # validation set entries: 14
[03.04|14:58:30] Storing data in step4_attempt1_reference_data
[03.04|14:58:30] Deleting step3_attempt4_reference_data
[03.04|14:58:30] Deleting step4_attempt1_reference_calc1
[03.04|14:58:30]
[03.04|14:58:30] Current (cumulative) timings:
[03.04|14:58:30] Time (s) Fraction
[03.04|14:58:30] Ref. calcs 224.53 0.122
[03.04|14:58:30] ML training 1232.16 0.668
[03.04|14:58:30] Simulations 389.19 0.211
[03.04|14:58:30]
[03.04|14:58:30]
[03.04|14:58:30]
[03.04|14:58:30] --- Begin summary ---
[03.04|14:58:30] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|14:58:30] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|14:58:30] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|14:58:30] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|14:58:30] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|14:58:30] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|14:58:30] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|14:58:30] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|14:58:30] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|14:58:30] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|14:58:30] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|14:58:30] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|14:58:30] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|14:58:30] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|14:58:30] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|14:58:30] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|14:58:30] --- End summary ---
[03.04|14:58:30]
[03.04|14:58:30] Running more reference calculations....
[03.04|14:58:30] Running reference calculations on frames [53, 71] from step4_attempt1_simulation/ams.rkf
[03.04|14:58:30] Calculating 2 frames in total
[03.04|14:58:30] Running step4_attempt1_reference_calc2
[03.04|14:58:38] Running step4_attempt1_reference_calc3
[03.04|14:58:45] Reference calculations finished!
[03.04|14:58:45] Adding results from step4_attempt1_reference_calc2 to validation set
[03.04|14:58:45] Adding results from step4_attempt1_reference_calc3 to training set
[03.04|14:58:45] Current # training set entries: 40
[03.04|14:58:45] Current # validation set entries: 15
[03.04|14:58:45] Storing data in step4_attempt1_reference_data
[03.04|14:58:45] Deleting step4_attempt1_reference_calc2
[03.04|14:58:45] Deleting step4_attempt1_reference_calc3
[03.04|14:58:45] Launching reparametrization job: step4_attempt1_training
[03.04|14:58:49] JOB optimizer_001 STARTED
[03.04|14:58:49] JOB optimizer_002 STARTED
[03.04|14:58:49] Starting optimizer_001.prerun()
[03.04|14:58:49] Starting optimizer_002.prerun()
[03.04|14:58:49] optimizer_001.prerun() finished
[03.04|14:58:49] optimizer_002.prerun() finished
[03.04|14:58:50] JOB optimizer_001 RUNNING
[03.04|14:58:50] JOB optimizer_002 RUNNING
[03.04|14:58:50] Executing optimizer_001.run
[03.04|14:58:50] Executing optimizer_002.run
[03.04|14:58:50] Waiting for job optimizer_001 to finish
[03.04|14:59:34] training_set Optimizer: 002 Epoch: 0 Loss: 0.005087
[03.04|14:59:34] training_set Optimizer: 001 Epoch: 0 Loss: 0.004974
[03.04|14:59:34] validation_set Optimizer: 002 Epoch: 0 Loss: 0.041754
[03.04|14:59:34] validation_set Optimizer: 001 Epoch: 0 Loss: 0.043684
[03.04|14:59:39] training_set Optimizer: 002 Epoch: 10 Loss: 0.002242
[03.04|14:59:39] training_set Optimizer: 001 Epoch: 10 Loss: 0.002299
[03.04|14:59:39] validation_set Optimizer: 001 Epoch: 10 Loss: 0.015929
[03.04|14:59:39] validation_set Optimizer: 002 Epoch: 10 Loss: 0.016429
[03.04|14:59:43] training_set Optimizer: 001 Epoch: 20 Loss: 0.002153
[03.04|14:59:43] training_set Optimizer: 002 Epoch: 20 Loss: 0.002424
[03.04|14:59:43] validation_set Optimizer: 001 Epoch: 20 Loss: 0.016829
[03.04|14:59:43] validation_set Optimizer: 002 Epoch: 20 Loss: 0.026385
[03.04|14:59:48] training_set Optimizer: 002 Epoch: 30 Loss: 0.002096
[03.04|14:59:48] training_set Optimizer: 001 Epoch: 30 Loss: 0.002274
[03.04|14:59:48] validation_set Optimizer: 002 Epoch: 30 Loss: 0.014644
[03.04|14:59:48] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014501
[03.04|14:59:52] training_set Optimizer: 002 Epoch: 40 Loss: 0.002260
[03.04|14:59:52] validation_set Optimizer: 002 Epoch: 40 Loss: 0.023164
[03.04|14:59:53] training_set Optimizer: 001 Epoch: 40 Loss: 0.002196
[03.04|14:59:53] validation_set Optimizer: 001 Epoch: 40 Loss: 0.013280
[03.04|14:59:57] training_set Optimizer: 002 Epoch: 50 Loss: 0.002134
[03.04|14:59:57] validation_set Optimizer: 002 Epoch: 50 Loss: 0.014741
[03.04|14:59:57] training_set Optimizer: 001 Epoch: 50 Loss: 0.002212
[03.04|14:59:57] validation_set Optimizer: 001 Epoch: 50 Loss: 0.011920
[03.04|15:00:01] training_set Optimizer: 002 Epoch: 60 Loss: 0.002127
[03.04|15:00:01] validation_set Optimizer: 002 Epoch: 60 Loss: 0.031417
[03.04|15:00:02] training_set Optimizer: 001 Epoch: 60 Loss: 0.002157
[03.04|15:00:02] validation_set Optimizer: 001 Epoch: 60 Loss: 0.014799
[03.04|15:00:06] training_set Optimizer: 002 Epoch: 70 Loss: 0.002016
[03.04|15:00:06] validation_set Optimizer: 002 Epoch: 70 Loss: 0.024824
[03.04|15:00:06] training_set Optimizer: 001 Epoch: 70 Loss: 0.002091
[03.04|15:00:06] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011481
[03.04|15:00:10] training_set Optimizer: 002 Epoch: 80 Loss: 0.001886
[03.04|15:00:10] validation_set Optimizer: 002 Epoch: 80 Loss: 0.018399
[03.04|15:00:11] training_set Optimizer: 001 Epoch: 80 Loss: 0.002087
[03.04|15:00:11] validation_set Optimizer: 001 Epoch: 80 Loss: 0.011138
[03.04|15:00:15] training_set Optimizer: 002 Epoch: 90 Loss: 0.001874
[03.04|15:00:15] validation_set Optimizer: 002 Epoch: 90 Loss: 0.018624
[03.04|15:00:15] training_set Optimizer: 001 Epoch: 90 Loss: 0.001908
[03.04|15:00:15] validation_set Optimizer: 001 Epoch: 90 Loss: 0.018505
[03.04|15:00:19] training_set Optimizer: 002 Epoch: 100 Loss: 0.001959
[03.04|15:00:19] validation_set Optimizer: 002 Epoch: 100 Loss: 0.017901
[03.04|15:00:20] training_set Optimizer: 001 Epoch: 100 Loss: 0.001935
[03.04|15:00:20] validation_set Optimizer: 001 Epoch: 100 Loss: 0.016786
[03.04|15:00:24] training_set Optimizer: 002 Epoch: 110 Loss: 0.001967
[03.04|15:00:24] validation_set Optimizer: 002 Epoch: 110 Loss: 0.019488
[03.04|15:00:24] training_set Optimizer: 001 Epoch: 110 Loss: 0.002066
[03.04|15:00:24] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011637
[03.04|15:00:29] Execution of optimizer_002.run finished with returncode 0
[03.04|15:00:29] JOB optimizer_002 FINISHED
[03.04|15:00:29] Starting optimizer_002.postrun()
[03.04|15:00:29] optimizer_002.postrun() finished
[03.04|15:00:29] JOB optimizer_002 SUCCESSFUL
[03.04|15:00:29] Execution of optimizer_001.run finished with returncode 0
[03.04|15:00:30] JOB optimizer_001 FINISHED
[03.04|15:00:30] Starting optimizer_001.postrun()
[03.04|15:00:30] optimizer_001.postrun() finished
[03.04|15:00:30] JOB optimizer_001 SUCCESSFUL
[03.04|15:00:30] PLAMS environment cleaned up successfully
[03.04|15:00:30] PLAMS run finished. Goodbye
[03.04|15:00:30] ParAMSResults
[03.04|15:00:30] Newly created parameter file/dir: step4_attempt1_training/results/optimization/optimizer_001/m3gnet
[03.04|15:00:30] Newly created parameter file/dir: step4_attempt1_training/results/optimization/optimizer_002/m3gnet
[03.04|15:00:30] Done!
[03.04|15:00:30] Deleting step3_attempt3_training
[03.04|15:00:30] ##########################
[03.04|15:00:30] ### Step 4 / Attempt 2 ###
[03.04|15:00:30] ##########################
[03.04|15:00:30] MD Steps: 547 (cumulative: 720)
[03.04|15:00:30] Current engine settings:
[03.04|15:00:30]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt1_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt1_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:00:30] Running step4_attempt2_simulation...
[03.04|15:01:12] Job step4_attempt2_simulation finished
[03.04|15:01:12] Deleting files that are no longer needed...
[03.04|15:01:12] Energy uncertainty for final frame of step4_attempt2_simulation: 0.1566 eV
[03.04|15:01:12] 0.0130 eV/atom
[03.04|15:01:12] Forces uncertainty for final frame of step4_attempt2_simulation: 0.1602 eV/angstrom
[03.04|15:01:13] Launching reference calculation
[03.04|15:01:20] Reference calculation finished!
[03.04|15:01:20] Checking success for step4_attempt2
[03.04|15:01:39] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:01:39] CheckEnergy: normalization coefficient = 12
[03.04|15:01:39] CheckEnergy: Actual Threshold
[03.04|15:01:39] CheckEnergy: dE/12 -0.0241 0.2000 OK!
[03.04|15:01:39] CheckEnergy: ddE/12 0.0038 0.0050 OK! (relative to step4_attempt1_simulation:MDStep720)
[03.04|15:01:39]
[03.04|15:01:39] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:01:39] CheckForces: ------------
[03.04|15:01:39] CheckForces: Reference job from step4_attempt2_reference_calc1
[03.04|15:01:39] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt2_simulation
[03.04|15:01:39] CheckForces: ------------
[03.04|15:01:39] CheckForces: Histogram of forces
[03.04|15:01:39] CheckForces: eV/Ang Ref Pred
[03.04|15:01:39] CheckForces: -2 0 0
[03.04|15:01:39] CheckForces: -1 19 20
[03.04|15:01:39] CheckForces: 0 17 15
[03.04|15:01:39] CheckForces: 1 0 1
[03.04|15:01:39] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:01:39] CheckForces: Force components with an error exceeding the threshold:
[03.04|15:01:39] CheckForces: Ref Pred Delta Threshold
[03.04|15:01:39] CheckForces: 0.82 0.18 0.64 0.54
[03.04|15:01:39] CheckForces: -0.38 0.18 0.56 0.52
[03.04|15:01:39] CheckForces: -0.72 -0.17 0.55 0.54
[03.04|15:01:39] CheckForces: -0.77 -0.19 0.58 0.54
[03.04|15:01:39] CheckForces: Maximum deviation: 0.645 eV/angstrom
[03.04|15:01:39] CheckForces: Actual Threshold
[03.04|15:01:39] CheckForces: # > thr. 4 0 Not OK!
[03.04|15:01:39] CheckForces: MAE 0.235 0.30 OK!
[03.04|15:01:39] CheckForces: R^2 0.501 0.40 OK!
[03.04|15:01:39] CheckForces: --------------------
[03.04|15:01:39]
[03.04|15:01:39] Adding results from step4_attempt2_reference_calc1 to training set
[03.04|15:01:39] Current # training set entries: 41
[03.04|15:01:39] Current # validation set entries: 15
[03.04|15:01:39] Storing data in step4_attempt2_reference_data
[03.04|15:01:39] Deleting step4_attempt1_reference_data
[03.04|15:01:39] Deleting step4_attempt2_reference_calc1
[03.04|15:01:39]
[03.04|15:01:39] Current (cumulative) timings:
[03.04|15:01:39] Time (s) Fraction
[03.04|15:01:39] Ref. calcs 246.59 0.122
[03.04|15:01:39] ML training 1337.28 0.664
[03.04|15:01:39] Simulations 431.03 0.214
[03.04|15:01:39]
[03.04|15:01:39]
[03.04|15:01:39]
[03.04|15:01:39] --- Begin summary ---
[03.04|15:01:39] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:01:39] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:01:39] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:01:39] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:01:39] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:01:39] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:01:39] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:01:39] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:01:39] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:01:39] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:01:39] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:01:39] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:01:39] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:01:39] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:01:39] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:01:39] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:01:39] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:01:39] --- End summary ---
[03.04|15:01:39]
[03.04|15:01:39] Running more reference calculations....
[03.04|15:01:39] Running reference calculations on frames [53, 71] from step4_attempt2_simulation/ams.rkf
[03.04|15:01:39] Calculating 2 frames in total
[03.04|15:01:39] Running step4_attempt2_reference_calc2
[03.04|15:01:46] Running step4_attempt2_reference_calc3
[03.04|15:01:53] Reference calculations finished!
[03.04|15:01:54] Adding results from step4_attempt2_reference_calc2 to validation set
[03.04|15:01:54] Adding results from step4_attempt2_reference_calc3 to training set
[03.04|15:01:54] Current # training set entries: 42
[03.04|15:01:54] Current # validation set entries: 16
[03.04|15:01:54] Storing data in step4_attempt2_reference_data
[03.04|15:01:54] Deleting step4_attempt2_reference_calc2
[03.04|15:01:54] Deleting step4_attempt2_reference_calc3
[03.04|15:01:54] Launching reparametrization job: step4_attempt2_training
[03.04|15:01:58] JOB optimizer_001 STARTED
[03.04|15:01:58] JOB optimizer_002 STARTED
[03.04|15:01:58] Starting optimizer_001.prerun()
[03.04|15:01:58] optimizer_001.prerun() finished
[03.04|15:01:58] Starting optimizer_002.prerun()
[03.04|15:01:58] optimizer_002.prerun() finished
[03.04|15:01:58] JOB optimizer_002 RUNNING
[03.04|15:01:58] Executing optimizer_002.run
[03.04|15:01:58] JOB optimizer_001 RUNNING
[03.04|15:01:58] Executing optimizer_001.run
[03.04|15:01:58] Waiting for job optimizer_001 to finish
[03.04|15:03:05] training_set Optimizer: 002 Epoch: 0 Loss: 0.004674
[03.04|15:03:05] validation_set Optimizer: 002 Epoch: 0 Loss: 0.044364
[03.04|15:03:06] training_set Optimizer: 001 Epoch: 0 Loss: 0.005075
[03.04|15:03:06] validation_set Optimizer: 001 Epoch: 0 Loss: 0.030040
[03.04|15:03:10] training_set Optimizer: 002 Epoch: 10 Loss: 0.001951
[03.04|15:03:10] validation_set Optimizer: 002 Epoch: 10 Loss: 0.022782
[03.04|15:03:11] training_set Optimizer: 001 Epoch: 10 Loss: 0.002058
[03.04|15:03:11] validation_set Optimizer: 001 Epoch: 10 Loss: 0.017983
[03.04|15:03:15] training_set Optimizer: 002 Epoch: 20 Loss: 0.002098
[03.04|15:03:15] validation_set Optimizer: 002 Epoch: 20 Loss: 0.018306
[03.04|15:03:16] training_set Optimizer: 001 Epoch: 20 Loss: 0.002324
[03.04|15:03:16] validation_set Optimizer: 001 Epoch: 20 Loss: 0.030950
[03.04|15:03:21] training_set Optimizer: 002 Epoch: 30 Loss: 0.002120
[03.04|15:03:21] validation_set Optimizer: 002 Epoch: 30 Loss: 0.030328
[03.04|15:03:21] training_set Optimizer: 001 Epoch: 30 Loss: 0.001907
[03.04|15:03:21] validation_set Optimizer: 001 Epoch: 30 Loss: 0.024062
[03.04|15:03:26] training_set Optimizer: 002 Epoch: 40 Loss: 0.002082
[03.04|15:03:26] validation_set Optimizer: 002 Epoch: 40 Loss: 0.023167
[03.04|15:03:26] training_set Optimizer: 001 Epoch: 40 Loss: 0.002637
[03.04|15:03:26] validation_set Optimizer: 001 Epoch: 40 Loss: 0.021514
[03.04|15:03:31] training_set Optimizer: 002 Epoch: 50 Loss: 0.002155
[03.04|15:03:31] validation_set Optimizer: 002 Epoch: 50 Loss: 0.025160
[03.04|15:03:32] training_set Optimizer: 001 Epoch: 50 Loss: 0.001819
[03.04|15:03:32] validation_set Optimizer: 001 Epoch: 50 Loss: 0.016034
[03.04|15:03:36] training_set Optimizer: 002 Epoch: 60 Loss: 0.001762
[03.04|15:03:36] validation_set Optimizer: 002 Epoch: 60 Loss: 0.018730
[03.04|15:03:37] training_set Optimizer: 001 Epoch: 60 Loss: 0.001794
[03.04|15:03:37] validation_set Optimizer: 001 Epoch: 60 Loss: 0.019620
[03.04|15:03:41] training_set Optimizer: 002 Epoch: 70 Loss: 0.001873
[03.04|15:03:41] validation_set Optimizer: 002 Epoch: 70 Loss: 0.029827
[03.04|15:03:42] training_set Optimizer: 001 Epoch: 70 Loss: 0.001803
[03.04|15:03:42] validation_set Optimizer: 001 Epoch: 70 Loss: 0.016135
[03.04|15:03:47] training_set Optimizer: 002 Epoch: 80 Loss: 0.001852
[03.04|15:03:47] validation_set Optimizer: 002 Epoch: 80 Loss: 0.018109
[03.04|15:03:47] training_set Optimizer: 001 Epoch: 80 Loss: 0.001758
[03.04|15:03:47] validation_set Optimizer: 001 Epoch: 80 Loss: 0.014654
[03.04|15:03:52] training_set Optimizer: 002 Epoch: 90 Loss: 0.002003
[03.04|15:03:52] validation_set Optimizer: 002 Epoch: 90 Loss: 0.035111
[03.04|15:03:52] training_set Optimizer: 001 Epoch: 90 Loss: 0.001952
[03.04|15:03:52] validation_set Optimizer: 001 Epoch: 90 Loss: 0.023219
[03.04|15:03:57] training_set Optimizer: 002 Epoch: 100 Loss: 0.001707
[03.04|15:03:57] validation_set Optimizer: 002 Epoch: 100 Loss: 0.017986
[03.04|15:03:58] training_set Optimizer: 001 Epoch: 100 Loss: 0.001778
[03.04|15:03:58] validation_set Optimizer: 001 Epoch: 100 Loss: 0.021165
[03.04|15:04:02] training_set Optimizer: 002 Epoch: 110 Loss: 0.001813
[03.04|15:04:02] validation_set Optimizer: 002 Epoch: 110 Loss: 0.014660
[03.04|15:04:03] training_set Optimizer: 001 Epoch: 110 Loss: 0.001846
[03.04|15:04:03] validation_set Optimizer: 001 Epoch: 110 Loss: 0.020958
[03.04|15:04:09] Execution of optimizer_002.run finished with returncode 0
[03.04|15:04:09] JOB optimizer_002 FINISHED
[03.04|15:04:09] Starting optimizer_002.postrun()
[03.04|15:04:09] optimizer_002.postrun() finished
[03.04|15:04:09] JOB optimizer_002 SUCCESSFUL
[03.04|15:04:09] Execution of optimizer_001.run finished with returncode 0
[03.04|15:04:09] JOB optimizer_001 FINISHED
[03.04|15:04:09] Starting optimizer_001.postrun()
[03.04|15:04:09] optimizer_001.postrun() finished
[03.04|15:04:09] JOB optimizer_001 SUCCESSFUL
[03.04|15:04:09] PLAMS environment cleaned up successfully
[03.04|15:04:09] PLAMS run finished. Goodbye
[03.04|15:04:10] ParAMSResults
[03.04|15:04:10] Newly created parameter file/dir: step4_attempt2_training/results/optimization/optimizer_001/m3gnet
[03.04|15:04:10] Newly created parameter file/dir: step4_attempt2_training/results/optimization/optimizer_002/m3gnet
[03.04|15:04:10] Done!
[03.04|15:04:10] Deleting step4_attempt1_training
[03.04|15:04:10] ##########################
[03.04|15:04:10] ### Step 4 / Attempt 3 ###
[03.04|15:04:10] ##########################
[03.04|15:04:10] MD Steps: 547 (cumulative: 720)
[03.04|15:04:10] Current engine settings:
[03.04|15:04:10]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:04:10] Running step4_attempt3_simulation...
[03.04|15:04:53] Job step4_attempt3_simulation finished
[03.04|15:04:53] Deleting files that are no longer needed...
[03.04|15:04:53] Energy uncertainty for final frame of step4_attempt3_simulation: 0.0957 eV
[03.04|15:04:53] 0.0080 eV/atom
[03.04|15:04:53] Forces uncertainty for final frame of step4_attempt3_simulation: 0.2679 eV/angstrom
[03.04|15:04:53] Launching reference calculation
[03.04|15:05:00] Reference calculation finished!
[03.04|15:05:00] Checking success for step4_attempt3
[03.04|15:05:20] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:05:20] CheckEnergy: normalization coefficient = 12
[03.04|15:05:20] CheckEnergy: Actual Threshold
[03.04|15:05:20] CheckEnergy: dE/12 -0.0149 0.2000 OK!
[03.04|15:05:20] CheckEnergy: ddE/12 -0.0001 0.0050 OK! (relative to step4_attempt2_simulation:MDStep720)
[03.04|15:05:20]
[03.04|15:05:20] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:05:20] CheckForces: ------------
[03.04|15:05:20] CheckForces: Reference job from step4_attempt3_reference_calc1
[03.04|15:05:20] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt3_simulation
[03.04|15:05:20] CheckForces: ------------
[03.04|15:05:20] CheckForces: Histogram of forces
[03.04|15:05:20] CheckForces: eV/Ang Ref Pred
[03.04|15:05:20] CheckForces: -2 0 0
[03.04|15:05:20] CheckForces: -1 16 19
[03.04|15:05:20] CheckForces: 0 20 16
[03.04|15:05:20] CheckForces: 1 0 1
[03.04|15:05:20] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:05:20] CheckForces: Force components with an error exceeding the threshold:
[03.04|15:05:20] CheckForces: Ref Pred Delta Threshold
[03.04|15:05:20] CheckForces: -0.59 -0.04 0.55 0.53
[03.04|15:05:20] CheckForces: -0.69 -0.12 0.57 0.53
[03.04|15:05:20] CheckForces: -0.70 -0.15 0.56 0.54
[03.04|15:05:20] CheckForces: Maximum deviation: 0.571 eV/angstrom
[03.04|15:05:20] CheckForces: Actual Threshold
[03.04|15:05:20] CheckForces: # > thr. 3 0 Not OK!
[03.04|15:05:20] CheckForces: MAE 0.207 0.30 OK!
[03.04|15:05:20] CheckForces: R^2 0.601 0.40 OK!
[03.04|15:05:20] CheckForces: --------------------
[03.04|15:05:20]
[03.04|15:05:20] Adding results from step4_attempt3_reference_calc1 to training set
[03.04|15:05:20] Current # training set entries: 43
[03.04|15:05:20] Current # validation set entries: 16
[03.04|15:05:20] Storing data in step4_attempt3_reference_data
[03.04|15:05:21] Deleting step4_attempt2_reference_data
[03.04|15:05:21] Deleting step4_attempt3_reference_calc1
[03.04|15:05:21]
[03.04|15:05:21] Current (cumulative) timings:
[03.04|15:05:21] Time (s) Fraction
[03.04|15:05:21] Ref. calcs 268.20 0.121
[03.04|15:05:21] ML training 1473.40 0.665
[03.04|15:05:21] Simulations 473.82 0.214
[03.04|15:05:21]
[03.04|15:05:21]
[03.04|15:05:21]
[03.04|15:05:21] --- Begin summary ---
[03.04|15:05:21] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:05:21] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:05:21] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:05:21] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:05:21] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:05:21] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:05:21] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:05:21] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:05:21] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:05:21] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:05:21] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:05:21] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:05:21] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:05:21] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:05:21] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:05:21] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:05:21] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:05:21] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:05:21] --- End summary ---
[03.04|15:05:21]
[03.04|15:05:21] Running more reference calculations....
[03.04|15:05:21] Running reference calculations on frames [53, 71] from step4_attempt3_simulation/ams.rkf
[03.04|15:05:21] Calculating 2 frames in total
[03.04|15:05:21] Running step4_attempt3_reference_calc2
[03.04|15:05:35] Running step4_attempt3_reference_calc3
[03.04|15:05:49] Reference calculations finished!
[03.04|15:05:49] Adding results from step4_attempt3_reference_calc2 to validation set
[03.04|15:05:49] Adding results from step4_attempt3_reference_calc3 to training set
[03.04|15:05:49] Current # training set entries: 44
[03.04|15:05:49] Current # validation set entries: 17
[03.04|15:05:49] Storing data in step4_attempt3_reference_data
[03.04|15:05:50] Deleting step4_attempt3_reference_calc2
[03.04|15:05:50] Deleting step4_attempt3_reference_calc3
[03.04|15:05:50] Launching reparametrization job: step4_attempt3_training
[03.04|15:05:54] JOB optimizer_001 STARTED
[03.04|15:05:54] JOB optimizer_002 STARTED
[03.04|15:05:54] Starting optimizer_001.prerun()
[03.04|15:05:54] Starting optimizer_002.prerun()
[03.04|15:05:54] optimizer_001.prerun() finished
[03.04|15:05:54] optimizer_002.prerun() finished
[03.04|15:05:54] JOB optimizer_001 RUNNING
[03.04|15:05:54] Executing optimizer_001.run
[03.04|15:05:54] JOB optimizer_002 RUNNING
[03.04|15:05:54] Executing optimizer_002.run
[03.04|15:05:54] Waiting for job optimizer_001 to finish
[03.04|15:07:12] training_set Optimizer: 001 Epoch: 0 Loss: 0.003474
[03.04|15:07:12] validation_set Optimizer: 001 Epoch: 0 Loss: 0.025083
[03.04|15:07:18] training_set Optimizer: 001 Epoch: 10 Loss: 0.001902
[03.04|15:07:18] validation_set Optimizer: 001 Epoch: 10 Loss: 0.023013
[03.04|15:07:23] training_set Optimizer: 001 Epoch: 20 Loss: 0.001824
[03.04|15:07:23] validation_set Optimizer: 001 Epoch: 20 Loss: 0.016186
[03.04|15:07:28] training_set Optimizer: 001 Epoch: 30 Loss: 0.002026
[03.04|15:07:28] validation_set Optimizer: 001 Epoch: 30 Loss: 0.021354
[03.04|15:07:33] training_set Optimizer: 002 Epoch: 0 Loss: 0.003201
[03.04|15:07:33] validation_set Optimizer: 002 Epoch: 0 Loss: 0.029701
[03.04|15:07:33] training_set Optimizer: 001 Epoch: 40 Loss: 0.001764
[03.04|15:07:33] validation_set Optimizer: 001 Epoch: 40 Loss: 0.021452
[03.04|15:07:39] training_set Optimizer: 002 Epoch: 10 Loss: 0.001874
[03.04|15:07:39] validation_set Optimizer: 002 Epoch: 10 Loss: 0.025366
[03.04|15:07:39] training_set Optimizer: 001 Epoch: 50 Loss: 0.001869
[03.04|15:07:39] validation_set Optimizer: 001 Epoch: 50 Loss: 0.017385
[03.04|15:07:45] training_set Optimizer: 002 Epoch: 20 Loss: 0.001933
[03.04|15:07:45] validation_set Optimizer: 002 Epoch: 20 Loss: 0.017794
[03.04|15:07:45] training_set Optimizer: 001 Epoch: 60 Loss: 0.001786
[03.04|15:07:45] validation_set Optimizer: 001 Epoch: 60 Loss: 0.017333
[03.04|15:07:51] training_set Optimizer: 002 Epoch: 30 Loss: 0.001788
[03.04|15:07:51] validation_set Optimizer: 002 Epoch: 30 Loss: 0.021722
[03.04|15:07:52] training_set Optimizer: 001 Epoch: 70 Loss: 0.001807
[03.04|15:07:52] validation_set Optimizer: 001 Epoch: 70 Loss: 0.026905
[03.04|15:07:57] training_set Optimizer: 002 Epoch: 40 Loss: 0.001793
[03.04|15:07:57] validation_set Optimizer: 002 Epoch: 40 Loss: 0.016738
[03.04|15:07:58] training_set Optimizer: 001 Epoch: 80 Loss: 0.001672
[03.04|15:07:58] validation_set Optimizer: 001 Epoch: 80 Loss: 0.015175
[03.04|15:08:03] training_set Optimizer: 002 Epoch: 50 Loss: 0.001723
[03.04|15:08:03] validation_set Optimizer: 002 Epoch: 50 Loss: 0.025286
[03.04|15:08:04] training_set Optimizer: 001 Epoch: 90 Loss: 0.001766
[03.04|15:08:04] validation_set Optimizer: 001 Epoch: 90 Loss: 0.014081
[03.04|15:08:09] training_set Optimizer: 002 Epoch: 60 Loss: 0.001746
[03.04|15:08:09] validation_set Optimizer: 002 Epoch: 60 Loss: 0.013514
[03.04|15:08:10] training_set Optimizer: 001 Epoch: 100 Loss: 0.001573
[03.04|15:08:10] validation_set Optimizer: 001 Epoch: 100 Loss: 0.014356
[03.04|15:08:15] training_set Optimizer: 002 Epoch: 70 Loss: 0.001745
[03.04|15:08:15] validation_set Optimizer: 002 Epoch: 70 Loss: 0.017987
[03.04|15:08:16] training_set Optimizer: 001 Epoch: 110 Loss: 0.001529
[03.04|15:08:16] validation_set Optimizer: 001 Epoch: 110 Loss: 0.015028
[03.04|15:08:21] training_set Optimizer: 002 Epoch: 80 Loss: 0.001644
[03.04|15:08:21] validation_set Optimizer: 002 Epoch: 80 Loss: 0.015397
[03.04|15:08:24] Execution of optimizer_001.run finished with returncode 0
[03.04|15:08:24] JOB optimizer_001 FINISHED
[03.04|15:08:24] Starting optimizer_001.postrun()
[03.04|15:08:24] optimizer_001.postrun() finished
[03.04|15:08:24] JOB optimizer_001 SUCCESSFUL
[03.04|15:08:24] Waiting for job optimizer_002 to finish
[03.04|15:08:26] training_set Optimizer: 002 Epoch: 90 Loss: 0.001721
[03.04|15:08:26] validation_set Optimizer: 002 Epoch: 90 Loss: 0.016904
[03.04|15:08:31] training_set Optimizer: 002 Epoch: 100 Loss: 0.001733
[03.04|15:08:31] validation_set Optimizer: 002 Epoch: 100 Loss: 0.020920
[03.04|15:08:36] training_set Optimizer: 002 Epoch: 110 Loss: 0.001667
[03.04|15:08:36] validation_set Optimizer: 002 Epoch: 110 Loss: 0.027802
[03.04|15:08:43] Execution of optimizer_002.run finished with returncode 0
[03.04|15:08:43] JOB optimizer_002 FINISHED
[03.04|15:08:43] Starting optimizer_002.postrun()
[03.04|15:08:43] optimizer_002.postrun() finished
[03.04|15:08:43] JOB optimizer_002 SUCCESSFUL
[03.04|15:08:43] PLAMS environment cleaned up successfully
[03.04|15:08:43] PLAMS run finished. Goodbye
[03.04|15:08:44] ParAMSResults
[03.04|15:08:44] Newly created parameter file/dir: step4_attempt3_training/results/optimization/optimizer_001/m3gnet
[03.04|15:08:44] Newly created parameter file/dir: step4_attempt3_training/results/optimization/optimizer_002/m3gnet
[03.04|15:08:44] Done!
[03.04|15:08:44] Deleting step4_attempt2_training
[03.04|15:08:44] ##########################
[03.04|15:08:44] ### Step 4 / Attempt 4 ###
[03.04|15:08:44] ##########################
[03.04|15:08:44] MD Steps: 547 (cumulative: 720)
[03.04|15:08:44] Current engine settings:
[03.04|15:08:44]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt3_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt3_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:08:44] Running step4_attempt4_simulation...
[03.04|15:09:35] Job step4_attempt4_simulation finished
[03.04|15:09:35] Deleting files that are no longer needed...
[03.04|15:09:35] Energy uncertainty for final frame of step4_attempt4_simulation: 0.0382 eV
[03.04|15:09:35] 0.0032 eV/atom
[03.04|15:09:35] Forces uncertainty for final frame of step4_attempt4_simulation: 0.3434 eV/angstrom
[03.04|15:09:35] Launching reference calculation
[03.04|15:09:52] Reference calculation finished!
[03.04|15:09:52] Checking success for step4_attempt4
[03.04|15:10:15] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:10:15] CheckEnergy: normalization coefficient = 12
[03.04|15:10:15] CheckEnergy: Actual Threshold
[03.04|15:10:15] CheckEnergy: dE/12 -0.0083 0.2000 OK!
[03.04|15:10:15] CheckEnergy: ddE/12 0.0090 0.0050 Not OK! (relative to step4_attempt3_simulation:MDStep720)
[03.04|15:10:15]
[03.04|15:10:15] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:10:15] CheckForces: ------------
[03.04|15:10:15] CheckForces: Reference job from step4_attempt4_reference_calc1
[03.04|15:10:15] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt4_simulation
[03.04|15:10:15] CheckForces: ------------
[03.04|15:10:15] CheckForces: Histogram of forces
[03.04|15:10:15] CheckForces: eV/Ang Ref Pred
[03.04|15:10:15] CheckForces: -2 0 0
[03.04|15:10:15] CheckForces: -1 19 18
[03.04|15:10:15] CheckForces: 0 15 17
[03.04|15:10:15] CheckForces: 1 2 1
[03.04|15:10:15] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:10:15] CheckForces: Force components with an error exceeding the threshold:
[03.04|15:10:15] CheckForces: Ref Pred Delta Threshold
[03.04|15:10:15] CheckForces: -0.48 0.10 0.58 0.52
[03.04|15:10:15] CheckForces: Maximum deviation: 0.584 eV/angstrom
[03.04|15:10:15] CheckForces: Actual Threshold
[03.04|15:10:15] CheckForces: # > thr. 1 0 Not OK!
[03.04|15:10:15] CheckForces: MAE 0.204 0.30 OK!
[03.04|15:10:15] CheckForces: R^2 0.697 0.40 OK!
[03.04|15:10:15] CheckForces: --------------------
[03.04|15:10:15]
[03.04|15:10:16] Adding results from step4_attempt4_reference_calc1 to training set
[03.04|15:10:16] Current # training set entries: 45
[03.04|15:10:16] Current # validation set entries: 17
[03.04|15:10:16] Storing data in step4_attempt4_reference_data
[03.04|15:10:16] Deleting step4_attempt3_reference_data
[03.04|15:10:16] Deleting step4_attempt4_reference_calc1
[03.04|15:10:16]
[03.04|15:10:16] Current (cumulative) timings:
[03.04|15:10:16] Time (s) Fraction
[03.04|15:10:16] Ref. calcs 313.91 0.126
[03.04|15:10:16] ML training 1648.04 0.663
[03.04|15:10:16] Simulations 524.40 0.211
[03.04|15:10:16]
[03.04|15:10:16]
[03.04|15:10:16]
[03.04|15:10:16] --- Begin summary ---
[03.04|15:10:16] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:10:16] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:10:16] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:10:16] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:10:16] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:10:16] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:10:16] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:10:16] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:10:16] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:10:16] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:10:16] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:10:16] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:10:16] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:10:16] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:10:16] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:10:16] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:10:16] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:10:16] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:10:16] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:10:16] --- End summary ---
[03.04|15:10:16]
[03.04|15:10:16] Running more reference calculations....
[03.04|15:10:16] Running reference calculations on frames [53, 71] from step4_attempt4_simulation/ams.rkf
[03.04|15:10:16] Calculating 2 frames in total
[03.04|15:10:16] Running step4_attempt4_reference_calc2
[03.04|15:10:34] Running step4_attempt4_reference_calc3
[03.04|15:10:51] Reference calculations finished!
[03.04|15:10:51] Adding results from step4_attempt4_reference_calc2 to validation set
[03.04|15:10:51] Adding results from step4_attempt4_reference_calc3 to training set
[03.04|15:10:51] Current # training set entries: 46
[03.04|15:10:51] Current # validation set entries: 18
[03.04|15:10:51] Storing data in step4_attempt4_reference_data
[03.04|15:10:52] Deleting step4_attempt4_reference_calc2
[03.04|15:10:52] Deleting step4_attempt4_reference_calc3
[03.04|15:10:52] Launching reparametrization job: step4_attempt4_training
[03.04|15:10:57] JOB optimizer_001 STARTED
[03.04|15:10:57] JOB optimizer_002 STARTED
[03.04|15:10:57] Starting optimizer_001.prerun()
[03.04|15:10:57] optimizer_001.prerun() finished
[03.04|15:10:57] Starting optimizer_002.prerun()
[03.04|15:10:57] optimizer_002.prerun() finished
[03.04|15:10:57] JOB optimizer_002 RUNNING
[03.04|15:10:57] JOB optimizer_001 RUNNING
[03.04|15:10:57] Executing optimizer_002.run
[03.04|15:10:57] Executing optimizer_001.run
[03.04|15:10:57] Waiting for job optimizer_001 to finish
[03.04|15:12:17] training_set Optimizer: 001 Epoch: 0 Loss: 0.005396
[03.04|15:12:17] validation_set Optimizer: 001 Epoch: 0 Loss: 0.017754
[03.04|15:12:18] training_set Optimizer: 002 Epoch: 0 Loss: 0.003322
[03.04|15:12:18] validation_set Optimizer: 002 Epoch: 0 Loss: 0.019030
[03.04|15:12:25] training_set Optimizer: 001 Epoch: 10 Loss: 0.001557
[03.04|15:12:25] validation_set Optimizer: 001 Epoch: 10 Loss: 0.022608
[03.04|15:12:26] training_set Optimizer: 002 Epoch: 10 Loss: 0.001470
[03.04|15:12:26] validation_set Optimizer: 002 Epoch: 10 Loss: 0.012070
[03.04|15:12:33] training_set Optimizer: 001 Epoch: 20 Loss: 0.001941
[03.04|15:12:33] validation_set Optimizer: 001 Epoch: 20 Loss: 0.015684
[03.04|15:12:34] training_set Optimizer: 002 Epoch: 20 Loss: 0.001653
[03.04|15:12:34] validation_set Optimizer: 002 Epoch: 20 Loss: 0.017705
[03.04|15:12:41] training_set Optimizer: 001 Epoch: 30 Loss: 0.001567
[03.04|15:12:41] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014466
[03.04|15:12:42] training_set Optimizer: 002 Epoch: 30 Loss: 0.001955
[03.04|15:12:42] validation_set Optimizer: 002 Epoch: 30 Loss: 0.018066
[03.04|15:12:48] training_set Optimizer: 001 Epoch: 40 Loss: 0.001567
[03.04|15:12:48] validation_set Optimizer: 001 Epoch: 40 Loss: 0.022485
[03.04|15:12:49] training_set Optimizer: 002 Epoch: 40 Loss: 0.001529
[03.04|15:12:49] validation_set Optimizer: 002 Epoch: 40 Loss: 0.011814
[03.04|15:12:55] training_set Optimizer: 001 Epoch: 50 Loss: 0.001477
[03.04|15:12:55] validation_set Optimizer: 001 Epoch: 50 Loss: 0.022205
[03.04|15:12:56] training_set Optimizer: 002 Epoch: 50 Loss: 0.001622
[03.04|15:12:56] validation_set Optimizer: 002 Epoch: 50 Loss: 0.013552
[03.04|15:13:02] training_set Optimizer: 001 Epoch: 60 Loss: 0.002095
[03.04|15:13:02] validation_set Optimizer: 001 Epoch: 60 Loss: 0.033039
[03.04|15:13:03] training_set Optimizer: 002 Epoch: 60 Loss: 0.001544
[03.04|15:13:03] validation_set Optimizer: 002 Epoch: 60 Loss: 0.031737
[03.04|15:13:08] training_set Optimizer: 001 Epoch: 70 Loss: 0.001577
[03.04|15:13:08] validation_set Optimizer: 001 Epoch: 70 Loss: 0.016890
[03.04|15:13:10] training_set Optimizer: 002 Epoch: 70 Loss: 0.001687
[03.04|15:13:10] validation_set Optimizer: 002 Epoch: 70 Loss: 0.030262
[03.04|15:13:16] training_set Optimizer: 001 Epoch: 80 Loss: 0.001710
[03.04|15:13:16] validation_set Optimizer: 001 Epoch: 80 Loss: 0.031683
[03.04|15:13:17] training_set Optimizer: 002 Epoch: 80 Loss: 0.001336
[03.04|15:13:17] validation_set Optimizer: 002 Epoch: 80 Loss: 0.017329
[03.04|15:13:22] training_set Optimizer: 001 Epoch: 90 Loss: 0.001660
[03.04|15:13:22] validation_set Optimizer: 001 Epoch: 90 Loss: 0.017519
[03.04|15:13:23] training_set Optimizer: 002 Epoch: 90 Loss: 0.001366
[03.04|15:13:23] validation_set Optimizer: 002 Epoch: 90 Loss: 0.014373
[03.04|15:13:29] training_set Optimizer: 001 Epoch: 100 Loss: 0.001401
[03.04|15:13:29] validation_set Optimizer: 001 Epoch: 100 Loss: 0.015601
[03.04|15:13:30] training_set Optimizer: 002 Epoch: 100 Loss: 0.001261
[03.04|15:13:30] validation_set Optimizer: 002 Epoch: 100 Loss: 0.026028
[03.04|15:13:35] training_set Optimizer: 001 Epoch: 110 Loss: 0.001513
[03.04|15:13:35] validation_set Optimizer: 001 Epoch: 110 Loss: 0.014411
[03.04|15:13:37] training_set Optimizer: 002 Epoch: 110 Loss: 0.001319
[03.04|15:13:37] validation_set Optimizer: 002 Epoch: 110 Loss: 0.013105
[03.04|15:13:44] Execution of optimizer_001.run finished with returncode 0
[03.04|15:13:44] JOB optimizer_001 FINISHED
[03.04|15:13:44] Starting optimizer_001.postrun()
[03.04|15:13:44] optimizer_001.postrun() finished
[03.04|15:13:44] JOB optimizer_001 SUCCESSFUL
[03.04|15:13:44] Waiting for job optimizer_002 to finish
[03.04|15:13:45] Execution of optimizer_002.run finished with returncode 0
[03.04|15:13:45] JOB optimizer_002 FINISHED
[03.04|15:13:45] Starting optimizer_002.postrun()
[03.04|15:13:45] optimizer_002.postrun() finished
[03.04|15:13:45] JOB optimizer_002 SUCCESSFUL
[03.04|15:13:45] PLAMS environment cleaned up successfully
[03.04|15:13:45] PLAMS run finished. Goodbye
[03.04|15:13:46] ParAMSResults
[03.04|15:13:46] Newly created parameter file/dir: step4_attempt4_training/results/optimization/optimizer_001/m3gnet
[03.04|15:13:46] Newly created parameter file/dir: step4_attempt4_training/results/optimization/optimizer_002/m3gnet
[03.04|15:13:46] Done!
[03.04|15:13:46] Deleting step4_attempt3_training
[03.04|15:13:46] ##########################
[03.04|15:13:46] ### Step 4 / Attempt 5 ###
[03.04|15:13:46] ##########################
[03.04|15:13:46] MD Steps: 547 (cumulative: 720)
[03.04|15:13:46] Current engine settings:
[03.04|15:13:46]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt4_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt4_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:13:46] Running step4_attempt5_simulation...
[03.04|15:14:37] Job step4_attempt5_simulation finished
[03.04|15:14:37] Deleting files that are no longer needed...
[03.04|15:14:37] Energy uncertainty for final frame of step4_attempt5_simulation: 0.2780 eV
[03.04|15:14:37] 0.0232 eV/atom
[03.04|15:14:37] Forces uncertainty for final frame of step4_attempt5_simulation: 0.6446 eV/angstrom
[03.04|15:14:37] Launching reference calculation
[03.04|15:14:52] Reference calculation finished!
[03.04|15:14:52] Checking success for step4_attempt5
[03.04|15:15:14] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:15:14] CheckEnergy: normalization coefficient = 12
[03.04|15:15:14] CheckEnergy: Actual Threshold
[03.04|15:15:14] CheckEnergy: dE/12 -0.0309 0.2000 OK!
[03.04|15:15:14] CheckEnergy: ddE/12 -0.0094 0.0050 Not OK! (relative to step4_attempt4_simulation:MDStep720)
[03.04|15:15:14]
[03.04|15:15:14] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:15:14] CheckForces: ------------
[03.04|15:15:14] CheckForces: Reference job from step4_attempt5_reference_calc1
[03.04|15:15:14] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt5_simulation
[03.04|15:15:14] CheckForces: ------------
[03.04|15:15:14] CheckForces: Histogram of forces
[03.04|15:15:14] CheckForces: eV/Ang Ref Pred
[03.04|15:15:14] CheckForces: -2 1 0
[03.04|15:15:14] CheckForces: -1 18 17
[03.04|15:15:14] CheckForces: 0 16 19
[03.04|15:15:14] CheckForces: 1 1 0
[03.04|15:15:14] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:15:14] CheckForces: Force components with an error exceeding the threshold:
[03.04|15:15:14] CheckForces: Ref Pred Delta Threshold
[03.04|15:15:14] CheckForces: -0.31 0.32 0.63 0.51
[03.04|15:15:14] CheckForces: -1.16 -0.33 0.83 0.57
[03.04|15:15:14] CheckForces: Maximum deviation: 0.830 eV/angstrom
[03.04|15:15:14] CheckForces: Actual Threshold
[03.04|15:15:14] CheckForces: # > thr. 2 0 Not OK!
[03.04|15:15:14] CheckForces: MAE 0.216 0.30 OK!
[03.04|15:15:14] CheckForces: R^2 0.492 0.40 OK!
[03.04|15:15:14] CheckForces: --------------------
[03.04|15:15:14]
[03.04|15:15:14] Adding results from step4_attempt5_reference_calc1 to training set
[03.04|15:15:14] Current # training set entries: 47
[03.04|15:15:14] Current # validation set entries: 18
[03.04|15:15:14] Storing data in step4_attempt5_reference_data
[03.04|15:15:14] Deleting step4_attempt4_reference_data
[03.04|15:15:14] Deleting step4_attempt5_reference_calc1
[03.04|15:15:14]
[03.04|15:15:14] Current (cumulative) timings:
[03.04|15:15:14] Time (s) Fraction
[03.04|15:15:14] Ref. calcs 364.05 0.132
[03.04|15:15:14] ML training 1822.61 0.660
[03.04|15:15:14] Simulations 574.95 0.208
[03.04|15:15:14]
[03.04|15:15:14]
[03.04|15:15:14]
[03.04|15:15:14] --- Begin summary ---
[03.04|15:15:14] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:15:14] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:15:14] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:15:14] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:15:14] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:15:14] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:15:14] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:15:14] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:15:14] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:15:14] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:15:14] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:15:14] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:15:14] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:15:14] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:15:14] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:15:14] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:15:14] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:15:14] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:15:14] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:15:14] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:15:14] --- End summary ---
[03.04|15:15:14]
[03.04|15:15:14] Running more reference calculations....
[03.04|15:15:15] Running reference calculations on frames [53, 71] from step4_attempt5_simulation/ams.rkf
[03.04|15:15:15] Calculating 2 frames in total
[03.04|15:15:15] Running step4_attempt5_reference_calc2
[03.04|15:15:29] Running step4_attempt5_reference_calc3
[03.04|15:15:43] Reference calculations finished!
[03.04|15:15:43] Adding results from step4_attempt5_reference_calc2 to validation set
[03.04|15:15:44] Adding results from step4_attempt5_reference_calc3 to training set
[03.04|15:15:44] Current # training set entries: 48
[03.04|15:15:44] Current # validation set entries: 19
[03.04|15:15:44] Storing data in step4_attempt5_reference_data
[03.04|15:15:44] Deleting step4_attempt5_reference_calc2
[03.04|15:15:44] Deleting step4_attempt5_reference_calc3
[03.04|15:15:44] Launching reparametrization job: step4_attempt5_training
[03.04|15:15:48] JOB optimizer_001 STARTED
[03.04|15:15:48] JOB optimizer_002 STARTED
[03.04|15:15:48] Starting optimizer_001.prerun()
[03.04|15:15:48] Starting optimizer_002.prerun()
[03.04|15:15:48] optimizer_001.prerun() finished
[03.04|15:15:48] optimizer_002.prerun() finished
[03.04|15:15:48] JOB optimizer_002 RUNNING
[03.04|15:15:48] JOB optimizer_001 RUNNING
[03.04|15:15:48] Executing optimizer_001.run
[03.04|15:15:48] Executing optimizer_002.run
[03.04|15:15:48] Waiting for job optimizer_001 to finish
[03.04|15:17:04] training_set Optimizer: 001 Epoch: 0 Loss: 0.004692
[03.04|15:17:04] validation_set Optimizer: 001 Epoch: 0 Loss: 0.032940
[03.04|15:17:04] training_set Optimizer: 002 Epoch: 0 Loss: 0.003620
[03.04|15:17:04] validation_set Optimizer: 002 Epoch: 0 Loss: 0.041359
[03.04|15:17:10] training_set Optimizer: 001 Epoch: 10 Loss: 0.001505
[03.04|15:17:10] validation_set Optimizer: 001 Epoch: 10 Loss: 0.014665
[03.04|15:17:10] training_set Optimizer: 002 Epoch: 10 Loss: 0.001327
[03.04|15:17:10] validation_set Optimizer: 002 Epoch: 10 Loss: 0.013566
[03.04|15:17:17] training_set Optimizer: 001 Epoch: 20 Loss: 0.001444
[03.04|15:17:17] validation_set Optimizer: 001 Epoch: 20 Loss: 0.013318
[03.04|15:17:17] training_set Optimizer: 002 Epoch: 20 Loss: 0.001290
[03.04|15:17:17] validation_set Optimizer: 002 Epoch: 20 Loss: 0.011453
[03.04|15:17:24] training_set Optimizer: 001 Epoch: 30 Loss: 0.001465
[03.04|15:17:24] validation_set Optimizer: 001 Epoch: 30 Loss: 0.015599
[03.04|15:17:24] training_set Optimizer: 002 Epoch: 30 Loss: 0.001322
[03.04|15:17:24] validation_set Optimizer: 002 Epoch: 30 Loss: 0.013967
[03.04|15:17:30] training_set Optimizer: 001 Epoch: 40 Loss: 0.001475
[03.04|15:17:30] validation_set Optimizer: 001 Epoch: 40 Loss: 0.016646
[03.04|15:17:30] training_set Optimizer: 002 Epoch: 40 Loss: 0.001238
[03.04|15:17:30] validation_set Optimizer: 002 Epoch: 40 Loss: 0.018467
[03.04|15:17:37] training_set Optimizer: 001 Epoch: 50 Loss: 0.001450
[03.04|15:17:37] validation_set Optimizer: 001 Epoch: 50 Loss: 0.017444
[03.04|15:17:37] training_set Optimizer: 002 Epoch: 50 Loss: 0.001478
[03.04|15:17:37] validation_set Optimizer: 002 Epoch: 50 Loss: 0.023132
[03.04|15:17:43] training_set Optimizer: 001 Epoch: 60 Loss: 0.001346
[03.04|15:17:43] validation_set Optimizer: 001 Epoch: 60 Loss: 0.017255
[03.04|15:17:43] training_set Optimizer: 002 Epoch: 60 Loss: 0.001255
[03.04|15:17:43] validation_set Optimizer: 002 Epoch: 60 Loss: 0.011874
[03.04|15:17:49] training_set Optimizer: 002 Epoch: 70 Loss: 0.001141
[03.04|15:17:49] validation_set Optimizer: 002 Epoch: 70 Loss: 0.012386
[03.04|15:17:50] training_set Optimizer: 001 Epoch: 70 Loss: 0.001483
[03.04|15:17:50] validation_set Optimizer: 001 Epoch: 70 Loss: 0.014434
[03.04|15:17:56] training_set Optimizer: 002 Epoch: 80 Loss: 0.001137
[03.04|15:17:56] validation_set Optimizer: 002 Epoch: 80 Loss: 0.022816
[03.04|15:17:57] training_set Optimizer: 001 Epoch: 80 Loss: 0.001268
[03.04|15:17:57] validation_set Optimizer: 001 Epoch: 80 Loss: 0.016034
[03.04|15:18:02] training_set Optimizer: 002 Epoch: 90 Loss: 0.001223
[03.04|15:18:02] validation_set Optimizer: 002 Epoch: 90 Loss: 0.011675
[03.04|15:18:03] training_set Optimizer: 001 Epoch: 90 Loss: 0.001193
[03.04|15:18:03] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013941
[03.04|15:18:09] training_set Optimizer: 002 Epoch: 100 Loss: 0.001331
[03.04|15:18:09] validation_set Optimizer: 002 Epoch: 100 Loss: 0.013470
[03.04|15:18:09] training_set Optimizer: 001 Epoch: 100 Loss: 0.001235
[03.04|15:18:09] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013669
[03.04|15:18:15] training_set Optimizer: 002 Epoch: 110 Loss: 0.001166
[03.04|15:18:15] validation_set Optimizer: 002 Epoch: 110 Loss: 0.009215
[03.04|15:18:16] training_set Optimizer: 001 Epoch: 110 Loss: 0.001379
[03.04|15:18:16] validation_set Optimizer: 001 Epoch: 110 Loss: 0.015860
[03.04|15:18:23] Execution of optimizer_002.run finished with returncode 0
[03.04|15:18:24] JOB optimizer_002 FINISHED
[03.04|15:18:24] Starting optimizer_002.postrun()
[03.04|15:18:24] optimizer_002.postrun() finished
[03.04|15:18:24] JOB optimizer_002 SUCCESSFUL
[03.04|15:18:24] Execution of optimizer_001.run finished with returncode 0
[03.04|15:18:24] JOB optimizer_001 FINISHED
[03.04|15:18:24] Starting optimizer_001.postrun()
[03.04|15:18:24] optimizer_001.postrun() finished
[03.04|15:18:24] JOB optimizer_001 SUCCESSFUL
[03.04|15:18:24] PLAMS environment cleaned up successfully
[03.04|15:18:24] PLAMS run finished. Goodbye
[03.04|15:18:25] ParAMSResults
[03.04|15:18:25] Newly created parameter file/dir: step4_attempt5_training/results/optimization/optimizer_001/m3gnet
[03.04|15:18:25] Newly created parameter file/dir: step4_attempt5_training/results/optimization/optimizer_002/m3gnet
[03.04|15:18:25] Done!
[03.04|15:18:25] Deleting step4_attempt4_training
[03.04|15:18:25] ##########################
[03.04|15:18:25] ### Step 4 / Attempt 6 ###
[03.04|15:18:25] ##########################
[03.04|15:18:25] MD Steps: 547 (cumulative: 720)
[03.04|15:18:25] Current engine settings:
[03.04|15:18:25]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt5_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt5_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:18:25] Running step4_attempt6_simulation...
[03.04|15:19:12] Job step4_attempt6_simulation finished
[03.04|15:19:12] Deleting files that are no longer needed...
[03.04|15:19:12] Energy uncertainty for final frame of step4_attempt6_simulation: 0.0524 eV
[03.04|15:19:12] 0.0044 eV/atom
[03.04|15:19:12] Forces uncertainty for final frame of step4_attempt6_simulation: 0.2465 eV/angstrom
[03.04|15:19:13] Launching reference calculation
[03.04|15:19:27] Reference calculation finished!
[03.04|15:19:27] Checking success for step4_attempt6
[03.04|15:19:48] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:19:48] CheckEnergy: normalization coefficient = 12
[03.04|15:19:48] CheckEnergy: Actual Threshold
[03.04|15:19:48] CheckEnergy: dE/12 0.0181 0.2000 OK!
[03.04|15:19:48] CheckEnergy: ddE/12 0.0083 0.0050 Not OK! (relative to step4_attempt5_simulation:MDStep720)
[03.04|15:19:48]
[03.04|15:19:48] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:19:48] CheckForces: ------------
[03.04|15:19:48] CheckForces: Reference job from step4_attempt6_reference_calc1
[03.04|15:19:48] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt6_simulation
[03.04|15:19:48] CheckForces: ------------
[03.04|15:19:48] CheckForces: Histogram of forces
[03.04|15:19:48] CheckForces: eV/Ang Ref Pred
[03.04|15:19:48] CheckForces: -2 0 0
[03.04|15:19:48] CheckForces: -1 14 15
[03.04|15:19:48] CheckForces: 0 22 21
[03.04|15:19:48] CheckForces: 1 0 0
[03.04|15:19:48] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:19:48] CheckForces: All force components are within the acceptable error!
[03.04|15:19:48] CheckForces: Maximum deviation: 0.291 eV/angstrom
[03.04|15:19:48] CheckForces: Actual Threshold
[03.04|15:19:48] CheckForces: # > thr. 0 0 OK!
[03.04|15:19:48] CheckForces: MAE 0.106 0.30 OK!
[03.04|15:19:48] CheckForces: R^2 0.804 0.40 OK!
[03.04|15:19:48] CheckForces: --------------------
[03.04|15:19:48]
[03.04|15:19:48] Adding results from step4_attempt6_reference_calc1 to training set
[03.04|15:19:48] Current # training set entries: 49
[03.04|15:19:48] Current # validation set entries: 19
[03.04|15:19:48] Storing data in step4_attempt6_reference_data
[03.04|15:19:48] Deleting step4_attempt5_reference_data
[03.04|15:19:48] Deleting step4_attempt6_reference_calc1
[03.04|15:19:48]
[03.04|15:19:48] Current (cumulative) timings:
[03.04|15:19:48] Time (s) Fraction
[03.04|15:19:48] Ref. calcs 406.83 0.135
[03.04|15:19:48] ML training 1984.08 0.659
[03.04|15:19:48] Simulations 622.09 0.206
[03.04|15:19:48]
[03.04|15:19:48]
[03.04|15:19:48]
[03.04|15:19:48] --- Begin summary ---
[03.04|15:19:48] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:19:48] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:19:48] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:19:48] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:19:48] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:19:48] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:19:48] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:19:48] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:19:48] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:19:48] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:19:48] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:19:48] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:19:48] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:19:48] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:19:48] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:19:48] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:19:48] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:19:48] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:19:48] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:19:48] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:19:48] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:19:48] --- End summary ---
[03.04|15:19:48]
[03.04|15:19:48] Running more reference calculations....
[03.04|15:19:48] Running reference calculations on frames [53, 71] from step4_attempt6_simulation/ams.rkf
[03.04|15:19:48] Calculating 2 frames in total
[03.04|15:19:48] Running step4_attempt6_reference_calc2
[03.04|15:20:02] Running step4_attempt6_reference_calc3
[03.04|15:20:16] Reference calculations finished!
[03.04|15:20:16] Adding results from step4_attempt6_reference_calc2 to validation set
[03.04|15:20:16] Adding results from step4_attempt6_reference_calc3 to training set
[03.04|15:20:16] Current # training set entries: 50
[03.04|15:20:16] Current # validation set entries: 20
[03.04|15:20:16] Storing data in step4_attempt6_reference_data
[03.04|15:20:16] Deleting step4_attempt6_reference_calc2
[03.04|15:20:16] Deleting step4_attempt6_reference_calc3
[03.04|15:20:16] Launching reparametrization job: step4_attempt6_training
[03.04|15:20:21] JOB optimizer_001 STARTED
[03.04|15:20:21] Starting optimizer_001.prerun()
[03.04|15:20:21] JOB optimizer_002 STARTED
[03.04|15:20:21] optimizer_001.prerun() finished
[03.04|15:20:21] Starting optimizer_002.prerun()
[03.04|15:20:21] optimizer_002.prerun() finished
[03.04|15:20:21] JOB optimizer_002 RUNNING
[03.04|15:20:21] Executing optimizer_002.run
[03.04|15:20:21] JOB optimizer_001 RUNNING
[03.04|15:20:21] Executing optimizer_001.run
[03.04|15:20:21] Waiting for job optimizer_001 to finish
[03.04|15:21:11] training_set Optimizer: 001 Epoch: 0 Loss: 0.005425
[03.04|15:21:11] validation_set Optimizer: 001 Epoch: 0 Loss: 0.043209
[03.04|15:21:11] training_set Optimizer: 002 Epoch: 0 Loss: 0.003742
[03.04|15:21:11] validation_set Optimizer: 002 Epoch: 0 Loss: 0.042892
[03.04|15:21:17] training_set Optimizer: 001 Epoch: 10 Loss: 0.001152
[03.04|15:21:17] validation_set Optimizer: 001 Epoch: 10 Loss: 0.011529
[03.04|15:21:18] training_set Optimizer: 002 Epoch: 10 Loss: 0.001044
[03.04|15:21:18] validation_set Optimizer: 002 Epoch: 10 Loss: 0.012071
[03.04|15:21:24] training_set Optimizer: 001 Epoch: 20 Loss: 0.001132
[03.04|15:21:24] validation_set Optimizer: 001 Epoch: 20 Loss: 0.014610
[03.04|15:21:24] training_set Optimizer: 002 Epoch: 20 Loss: 0.001126
[03.04|15:21:24] validation_set Optimizer: 002 Epoch: 20 Loss: 0.015262
[03.04|15:21:30] training_set Optimizer: 001 Epoch: 30 Loss: 0.001094
[03.04|15:21:30] validation_set Optimizer: 001 Epoch: 30 Loss: 0.012522
[03.04|15:21:31] training_set Optimizer: 002 Epoch: 30 Loss: 0.000981
[03.04|15:21:31] validation_set Optimizer: 002 Epoch: 30 Loss: 0.013680
[03.04|15:21:37] training_set Optimizer: 001 Epoch: 40 Loss: 0.001103
[03.04|15:21:37] validation_set Optimizer: 001 Epoch: 40 Loss: 0.012214
[03.04|15:21:38] training_set Optimizer: 002 Epoch: 40 Loss: 0.001108
[03.04|15:21:38] validation_set Optimizer: 002 Epoch: 40 Loss: 0.012292
[03.04|15:21:44] training_set Optimizer: 002 Epoch: 50 Loss: 0.001000
[03.04|15:21:44] validation_set Optimizer: 002 Epoch: 50 Loss: 0.013543
[03.04|15:21:44] training_set Optimizer: 001 Epoch: 50 Loss: 0.001088
[03.04|15:21:44] validation_set Optimizer: 001 Epoch: 50 Loss: 0.013507
[03.04|15:21:50] training_set Optimizer: 001 Epoch: 60 Loss: 0.001174
[03.04|15:21:50] validation_set Optimizer: 001 Epoch: 60 Loss: 0.011616
[03.04|15:21:50] training_set Optimizer: 002 Epoch: 60 Loss: 0.001110
[03.04|15:21:50] validation_set Optimizer: 002 Epoch: 60 Loss: 0.015552
[03.04|15:21:56] training_set Optimizer: 001 Epoch: 70 Loss: 0.001023
[03.04|15:21:56] validation_set Optimizer: 001 Epoch: 70 Loss: 0.013209
[03.04|15:21:57] training_set Optimizer: 002 Epoch: 70 Loss: 0.001025
[03.04|15:21:57] validation_set Optimizer: 002 Epoch: 70 Loss: 0.010560
[03.04|15:22:03] training_set Optimizer: 001 Epoch: 80 Loss: 0.001225
[03.04|15:22:03] validation_set Optimizer: 001 Epoch: 80 Loss: 0.017161
[03.04|15:22:03] training_set Optimizer: 002 Epoch: 80 Loss: 0.001037
[03.04|15:22:03] validation_set Optimizer: 002 Epoch: 80 Loss: 0.017466
[03.04|15:22:10] training_set Optimizer: 001 Epoch: 90 Loss: 0.000953
[03.04|15:22:10] validation_set Optimizer: 001 Epoch: 90 Loss: 0.020960
[03.04|15:22:10] training_set Optimizer: 002 Epoch: 90 Loss: 0.001121
[03.04|15:22:10] validation_set Optimizer: 002 Epoch: 90 Loss: 0.014894
[03.04|15:22:16] training_set Optimizer: 001 Epoch: 100 Loss: 0.000982
[03.04|15:22:16] validation_set Optimizer: 001 Epoch: 100 Loss: 0.014635
[03.04|15:22:17] training_set Optimizer: 002 Epoch: 100 Loss: 0.001046
[03.04|15:22:17] validation_set Optimizer: 002 Epoch: 100 Loss: 0.009838
[03.04|15:22:23] training_set Optimizer: 001 Epoch: 110 Loss: 0.000928
[03.04|15:22:23] validation_set Optimizer: 001 Epoch: 110 Loss: 0.013510
[03.04|15:22:23] training_set Optimizer: 002 Epoch: 110 Loss: 0.000871
[03.04|15:22:23] validation_set Optimizer: 002 Epoch: 110 Loss: 0.009575
[03.04|15:22:30] Execution of optimizer_001.run finished with returncode 0
[03.04|15:22:30] JOB optimizer_001 FINISHED
[03.04|15:22:30] Starting optimizer_001.postrun()
[03.04|15:22:30] optimizer_001.postrun() finished
[03.04|15:22:30] JOB optimizer_001 SUCCESSFUL
[03.04|15:22:30] Waiting for job optimizer_002 to finish
[03.04|15:22:30] Execution of optimizer_002.run finished with returncode 0
[03.04|15:22:31] JOB optimizer_002 FINISHED
[03.04|15:22:31] Starting optimizer_002.postrun()
[03.04|15:22:31] optimizer_002.postrun() finished
[03.04|15:22:31] JOB optimizer_002 SUCCESSFUL
[03.04|15:22:31] PLAMS environment cleaned up successfully
[03.04|15:22:31] PLAMS run finished. Goodbye
[03.04|15:22:31] ParAMSResults
[03.04|15:22:31] Newly created parameter file/dir: step4_attempt6_training/results/optimization/optimizer_001/m3gnet
[03.04|15:22:31] Newly created parameter file/dir: step4_attempt6_training/results/optimization/optimizer_002/m3gnet
[03.04|15:22:31] Done!
[03.04|15:22:31] Deleting step4_attempt5_training
[03.04|15:22:31] ##########################
[03.04|15:22:31] ### Step 4 / Attempt 7 ###
[03.04|15:22:31] ##########################
[03.04|15:22:31] MD Steps: 547 (cumulative: 720)
[03.04|15:22:31] Current engine settings:
[03.04|15:22:31]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt6_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt6_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:22:31] Running step4_attempt7_simulation...
[03.04|15:23:19] Job step4_attempt7_simulation finished
[03.04|15:23:19] Deleting files that are no longer needed...
[03.04|15:23:19] Energy uncertainty for final frame of step4_attempt7_simulation: 0.0933 eV
[03.04|15:23:19] 0.0078 eV/atom
[03.04|15:23:19] Forces uncertainty for final frame of step4_attempt7_simulation: 0.2864 eV/angstrom
[03.04|15:23:19] Launching reference calculation
[03.04|15:23:33] Reference calculation finished!
[03.04|15:23:33] Checking success for step4_attempt7
[03.04|15:23:54] CheckEnergy: Checking energy for MDStep720, n_atoms = 12
[03.04|15:23:54] CheckEnergy: normalization coefficient = 12
[03.04|15:23:54] CheckEnergy: Actual Threshold
[03.04|15:23:54] CheckEnergy: dE/12 0.0013 0.2000 OK!
[03.04|15:23:54] CheckEnergy: ddE/12 0.0010 0.0050 OK! (relative to step4_attempt6_simulation:MDStep720)
[03.04|15:23:54]
[03.04|15:23:54] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|15:23:54] CheckForces: ------------
[03.04|15:23:54] CheckForces: Reference job from step4_attempt7_reference_calc1
[03.04|15:23:54] CheckForces: Prediction job from final frame (MDStep720) of step4_attempt7_simulation
[03.04|15:23:54] CheckForces: ------------
[03.04|15:23:54] CheckForces: Histogram of forces
[03.04|15:23:54] CheckForces: eV/Ang Ref Pred
[03.04|15:23:54] CheckForces: -2 0 0
[03.04|15:23:54] CheckForces: -1 16 18
[03.04|15:23:54] CheckForces: 0 20 18
[03.04|15:23:54] CheckForces: 1 0 0
[03.04|15:23:54] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|15:23:54] CheckForces: All force components are within the acceptable error!
[03.04|15:23:54] CheckForces: Maximum deviation: 0.314 eV/angstrom
[03.04|15:23:54] CheckForces: Actual Threshold
[03.04|15:23:54] CheckForces: # > thr. 0 0 OK!
[03.04|15:23:54] CheckForces: MAE 0.108 0.30 OK!
[03.04|15:23:54] CheckForces: R^2 0.805 0.40 OK!
[03.04|15:23:54] CheckForces: --------------------
[03.04|15:23:54]
[03.04|15:23:54] Adding results from step4_attempt7_reference_calc1 to validation set
[03.04|15:23:54] Current # training set entries: 50
[03.04|15:23:54] Current # validation set entries: 21
[03.04|15:23:54] Storing data in step4_attempt7_reference_data
[03.04|15:23:55] Deleting step4_attempt6_reference_data
[03.04|15:23:55] Deleting step4_attempt7_reference_calc1
[03.04|15:23:55]
[03.04|15:23:55] Current (cumulative) timings:
[03.04|15:23:55] Time (s) Fraction
[03.04|15:23:55] Ref. calcs 448.60 0.139
[03.04|15:23:55] ML training 2119.47 0.655
[03.04|15:23:55] Simulations 669.17 0.207
[03.04|15:23:55]
[03.04|15:23:55]
[03.04|15:23:55] Step 4 finished successfully!
[03.04|15:23:55]
[03.04|15:23:55] --- Begin summary ---
[03.04|15:23:55] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:23:55] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:23:55] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:23:55] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:23:55] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:23:55] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:23:55] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:23:55] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:23:55] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:23:55] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:23:55] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:23:55] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:23:55] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:23:55] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:23:55] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:23:55] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:23:55] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:23:55] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:23:55] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:23:55] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:23:55] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:23:55] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:23:55] --- End summary ---
[03.04|15:23:55]
[03.04|15:23:55] ##########################
[03.04|15:23:55] ### Step 5 / Attempt 1 ###
[03.04|15:23:55] ##########################
[03.04|15:23:55] MD Steps: 2280 (cumulative: 3000)
[03.04|15:23:55] Current engine settings:
[03.04|15:23:55]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt6_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step4_attempt6_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:23:55] Running step5_attempt1_simulation...
[03.04|15:24:35] Job step5_attempt1_simulation finished
[03.04|15:24:35] Deleting files that are no longer needed...
[03.04|15:24:35] Deleting step3_attempt4_simulation
[03.04|15:24:35] Deleting step4_attempt1_simulation
[03.04|15:24:35] Deleting step4_attempt2_simulation
[03.04|15:24:35] Deleting step4_attempt3_simulation
[03.04|15:24:35] Deleting step4_attempt4_simulation
[03.04|15:24:35] Deleting step4_attempt5_simulation
[03.04|15:24:35] Deleting step4_attempt6_simulation
[03.04|15:24:35] Energy uncertainty for final frame of step5_attempt1_simulation: 0.2926 eV
[03.04|15:24:35] 0.0244 eV/atom
[03.04|15:24:35] Forces uncertainty for final frame of step5_attempt1_simulation: 1.0061 eV/angstrom
[03.04|15:24:35] Launching reference calculation
[03.04|15:24:49] Reference calculation finished!
[03.04|15:24:49] Adding results from step5_attempt1_reference_calc1 to training set
[03.04|15:24:49] Current # training set entries: 51
[03.04|15:24:49] Current # validation set entries: 21
[03.04|15:24:49] Storing data in step5_attempt1_reference_data
[03.04|15:24:49] Deleting step4_attempt7_reference_data
[03.04|15:24:49] Deleting step5_attempt1_reference_calc1
[03.04|15:24:49]
[03.04|15:24:49] Current (cumulative) timings:
[03.04|15:24:49] Time (s) Fraction
[03.04|15:24:49] Ref. calcs 462.24 0.140
[03.04|15:24:49] ML training 2119.47 0.644
[03.04|15:24:49] Simulations 709.24 0.216
[03.04|15:24:49]
[03.04|15:24:49]
[03.04|15:24:49]
[03.04|15:24:49] --- Begin summary ---
[03.04|15:24:49] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:24:49] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:24:49] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:24:49] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:24:49] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:24:49] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:24:49] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:24:49] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:24:49] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:24:49] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:24:49] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:24:49] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:24:49] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:24:49] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:24:49] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:24:49] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:24:49] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:24:49] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:24:49] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:24:49] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:24:49] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:24:49] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:24:49] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:24:49] --- End summary ---
[03.04|15:24:49]
[03.04|15:24:49] Running more reference calculations....
[03.04|15:24:50] Running reference calculations on frames [106, 115] from step5_attempt1_simulation/ams.rkf
[03.04|15:24:50] Calculating 2 frames in total
[03.04|15:24:50] Running step5_attempt1_reference_calc2
[03.04|15:25:03] Running step5_attempt1_reference_calc3
[03.04|15:25:17] Reference calculations finished!
[03.04|15:25:18] Adding results from step5_attempt1_reference_calc2 to training set
[03.04|15:25:18] Adding results from step5_attempt1_reference_calc3 to training set
[03.04|15:25:18] Current # training set entries: 53
[03.04|15:25:18] Current # validation set entries: 21
[03.04|15:25:18] Storing data in step5_attempt1_reference_data
[03.04|15:25:18] Deleting step5_attempt1_reference_calc2
[03.04|15:25:18] Deleting step5_attempt1_reference_calc3
[03.04|15:25:18] Launching reparametrization job: step5_attempt1_training
[03.04|15:25:23] JOB optimizer_001 STARTED
[03.04|15:25:23] JOB optimizer_002 STARTED
[03.04|15:25:23] Starting optimizer_001.prerun()
[03.04|15:25:23] Starting optimizer_002.prerun()
[03.04|15:25:23] optimizer_002.prerun() finished
[03.04|15:25:23] optimizer_001.prerun() finished
[03.04|15:25:23] JOB optimizer_001 RUNNING
[03.04|15:25:23] JOB optimizer_002 RUNNING
[03.04|15:25:23] Executing optimizer_002.run
[03.04|15:25:23] Executing optimizer_001.run
[03.04|15:25:23] Waiting for job optimizer_001 to finish
[03.04|15:26:37] training_set Optimizer: 002 Epoch: 0 Loss: 0.003876
[03.04|15:26:37] validation_set Optimizer: 002 Epoch: 0 Loss: 0.024652
[03.04|15:26:37] training_set Optimizer: 001 Epoch: 0 Loss: 0.003587
[03.04|15:26:37] validation_set Optimizer: 001 Epoch: 0 Loss: 0.023401
[03.04|15:26:45] training_set Optimizer: 002 Epoch: 10 Loss: 0.001237
[03.04|15:26:45] validation_set Optimizer: 002 Epoch: 10 Loss: 0.012631
[03.04|15:26:45] training_set Optimizer: 001 Epoch: 10 Loss: 0.001274
[03.04|15:26:45] validation_set Optimizer: 001 Epoch: 10 Loss: 0.014842
[03.04|15:26:52] training_set Optimizer: 001 Epoch: 20 Loss: 0.001281
[03.04|15:26:52] validation_set Optimizer: 001 Epoch: 20 Loss: 0.018001
[03.04|15:26:52] training_set Optimizer: 002 Epoch: 20 Loss: 0.001064
[03.04|15:26:52] validation_set Optimizer: 002 Epoch: 20 Loss: 0.013630
[03.04|15:26:59] training_set Optimizer: 002 Epoch: 30 Loss: 0.001305
[03.04|15:26:59] validation_set Optimizer: 002 Epoch: 30 Loss: 0.018955
[03.04|15:26:59] training_set Optimizer: 001 Epoch: 30 Loss: 0.001273
[03.04|15:26:59] validation_set Optimizer: 001 Epoch: 30 Loss: 0.016539
[03.04|15:27:07] training_set Optimizer: 002 Epoch: 40 Loss: 0.001074
[03.04|15:27:07] validation_set Optimizer: 002 Epoch: 40 Loss: 0.019168
[03.04|15:27:07] training_set Optimizer: 001 Epoch: 40 Loss: 0.001173
[03.04|15:27:07] validation_set Optimizer: 001 Epoch: 40 Loss: 0.018626
[03.04|15:27:14] training_set Optimizer: 002 Epoch: 50 Loss: 0.001182
[03.04|15:27:14] validation_set Optimizer: 002 Epoch: 50 Loss: 0.018518
[03.04|15:27:14] training_set Optimizer: 001 Epoch: 50 Loss: 0.001094
[03.04|15:27:14] validation_set Optimizer: 001 Epoch: 50 Loss: 0.014704
[03.04|15:27:21] training_set Optimizer: 002 Epoch: 60 Loss: 0.001069
[03.04|15:27:21] validation_set Optimizer: 002 Epoch: 60 Loss: 0.025761
[03.04|15:27:21] training_set Optimizer: 001 Epoch: 60 Loss: 0.001043
[03.04|15:27:21] validation_set Optimizer: 001 Epoch: 60 Loss: 0.012618
[03.04|15:27:28] training_set Optimizer: 001 Epoch: 70 Loss: 0.000940
[03.04|15:27:28] validation_set Optimizer: 001 Epoch: 70 Loss: 0.018057
[03.04|15:27:28] training_set Optimizer: 002 Epoch: 70 Loss: 0.001208
[03.04|15:27:28] validation_set Optimizer: 002 Epoch: 70 Loss: 0.018616
[03.04|15:27:35] training_set Optimizer: 002 Epoch: 80 Loss: 0.001032
[03.04|15:27:35] validation_set Optimizer: 002 Epoch: 80 Loss: 0.018311
[03.04|15:27:35] training_set Optimizer: 001 Epoch: 80 Loss: 0.001068
[03.04|15:27:35] validation_set Optimizer: 001 Epoch: 80 Loss: 0.014590
[03.04|15:27:42] training_set Optimizer: 002 Epoch: 90 Loss: 0.000849
[03.04|15:27:42] validation_set Optimizer: 002 Epoch: 90 Loss: 0.020231
[03.04|15:27:43] training_set Optimizer: 001 Epoch: 90 Loss: 0.001003
[03.04|15:27:43] validation_set Optimizer: 001 Epoch: 90 Loss: 0.017973
[03.04|15:27:50] training_set Optimizer: 002 Epoch: 100 Loss: 0.000926
[03.04|15:27:50] validation_set Optimizer: 002 Epoch: 100 Loss: 0.017142
[03.04|15:27:50] training_set Optimizer: 001 Epoch: 100 Loss: 0.000968
[03.04|15:27:50] validation_set Optimizer: 001 Epoch: 100 Loss: 0.012803
[03.04|15:27:57] training_set Optimizer: 002 Epoch: 110 Loss: 0.000927
[03.04|15:27:57] validation_set Optimizer: 002 Epoch: 110 Loss: 0.018007
[03.04|15:27:57] training_set Optimizer: 001 Epoch: 110 Loss: 0.000937
[03.04|15:27:57] validation_set Optimizer: 001 Epoch: 110 Loss: 0.016638
[03.04|15:28:06] Execution of optimizer_002.run finished with returncode 0
[03.04|15:28:06] JOB optimizer_002 FINISHED
[03.04|15:28:06] Starting optimizer_002.postrun()
[03.04|15:28:06] optimizer_002.postrun() finished
[03.04|15:28:06] Execution of optimizer_001.run finished with returncode 0
[03.04|15:28:06] JOB optimizer_002 SUCCESSFUL
[03.04|15:28:06] JOB optimizer_001 FINISHED
[03.04|15:28:06] Starting optimizer_001.postrun()
[03.04|15:28:06] optimizer_001.postrun() finished
[03.04|15:28:07] JOB optimizer_001 SUCCESSFUL
[03.04|15:28:07] PLAMS environment cleaned up successfully
[03.04|15:28:07] PLAMS run finished. Goodbye
[03.04|15:28:07] ParAMSResults
[03.04|15:28:07] Newly created parameter file/dir: step5_attempt1_training/results/optimization/optimizer_001/m3gnet
[03.04|15:28:07] Newly created parameter file/dir: step5_attempt1_training/results/optimization/optimizer_002/m3gnet
[03.04|15:28:07] Done!
[03.04|15:28:07] Deleting step4_attempt6_training
[03.04|15:28:07] ##########################
[03.04|15:28:07] ### Step 5 / Attempt 2 ###
[03.04|15:28:07] ##########################
[03.04|15:28:07] MD Steps: 2280 (cumulative: 3000)
[03.04|15:28:07] Current engine settings:
[03.04|15:28:07]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt1_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt1_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:28:07] Running step5_attempt2_simulation...
[03.04|15:29:08] Job step5_attempt2_simulation finished
[03.04|15:29:08] Deleting files that are no longer needed...
[03.04|15:29:08] Energy uncertainty for final frame of step5_attempt2_simulation: 0.0179 eV
[03.04|15:29:08] 0.0015 eV/atom
[03.04|15:29:08] Forces uncertainty for final frame of step5_attempt2_simulation: 1.0275 eV/angstrom
[03.04|15:29:09] Launching reference calculation
[03.04|15:29:23] Reference calculation finished!
[03.04|15:29:23] Adding results from step5_attempt2_reference_calc1 to training set
[03.04|15:29:23] Current # training set entries: 54
[03.04|15:29:23] Current # validation set entries: 21
[03.04|15:29:23] Storing data in step5_attempt2_reference_data
[03.04|15:29:23] Deleting step5_attempt1_reference_data
[03.04|15:29:23] Deleting step5_attempt2_reference_calc1
[03.04|15:29:23]
[03.04|15:29:23] Current (cumulative) timings:
[03.04|15:29:23] Time (s) Fraction
[03.04|15:29:23] Ref. calcs 504.32 0.142
[03.04|15:29:23] ML training 2288.74 0.642
[03.04|15:29:23] Simulations 770.54 0.216
[03.04|15:29:23]
[03.04|15:29:23]
[03.04|15:29:23]
[03.04|15:29:23] --- Begin summary ---
[03.04|15:29:23] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:29:23] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:29:23] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:29:23] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:29:23] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:29:23] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:29:23] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:29:23] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:29:23] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:29:23] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:29:23] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:29:23] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:29:23] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:29:23] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:29:23] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:29:23] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:29:23] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:29:23] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:29:23] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:29:23] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:29:23] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:29:23] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:29:23] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:29:23] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:29:23] --- End summary ---
[03.04|15:29:23]
[03.04|15:29:23] Running more reference calculations....
[03.04|15:29:24] Running reference calculations on frames [170, 210] from step5_attempt2_simulation/ams.rkf
[03.04|15:29:24] Calculating 2 frames in total
[03.04|15:29:24] Running step5_attempt2_reference_calc2
[03.04|15:29:38] Running step5_attempt2_reference_calc3
[03.04|15:29:52] Reference calculations finished!
[03.04|15:29:52] Adding results from step5_attempt2_reference_calc2 to training set
[03.04|15:29:52] Adding results from step5_attempt2_reference_calc3 to training set
[03.04|15:29:52] Current # training set entries: 56
[03.04|15:29:52] Current # validation set entries: 21
[03.04|15:29:52] Storing data in step5_attempt2_reference_data
[03.04|15:29:52] Deleting step5_attempt2_reference_calc2
[03.04|15:29:52] Deleting step5_attempt2_reference_calc3
[03.04|15:29:52] Launching reparametrization job: step5_attempt2_training
[03.04|15:29:57] JOB optimizer_001 STARTED
[03.04|15:29:57] JOB optimizer_002 STARTED
[03.04|15:29:57] Starting optimizer_001.prerun()
[03.04|15:29:57] Starting optimizer_002.prerun()
[03.04|15:29:57] optimizer_001.prerun() finished
[03.04|15:29:57] optimizer_002.prerun() finished
[03.04|15:29:57] JOB optimizer_002 RUNNING
[03.04|15:29:57] JOB optimizer_001 RUNNING
[03.04|15:29:57] Executing optimizer_002.run
[03.04|15:29:57] Executing optimizer_001.run
[03.04|15:29:57] Waiting for job optimizer_001 to finish
[03.04|15:31:12] training_set Optimizer: 001 Epoch: 0 Loss: 0.008800
[03.04|15:31:12] validation_set Optimizer: 001 Epoch: 0 Loss: 0.023654
[03.04|15:31:12] training_set Optimizer: 002 Epoch: 0 Loss: 0.005826
[03.04|15:31:12] validation_set Optimizer: 002 Epoch: 0 Loss: 0.014558
[03.04|15:31:20] training_set Optimizer: 001 Epoch: 10 Loss: 0.002103
[03.04|15:31:20] validation_set Optimizer: 001 Epoch: 10 Loss: 0.016584
[03.04|15:31:20] training_set Optimizer: 002 Epoch: 10 Loss: 0.001943
[03.04|15:31:20] validation_set Optimizer: 002 Epoch: 10 Loss: 0.014571
[03.04|15:31:28] training_set Optimizer: 002 Epoch: 20 Loss: 0.001569
[03.04|15:31:28] validation_set Optimizer: 002 Epoch: 20 Loss: 0.012548
[03.04|15:31:28] training_set Optimizer: 001 Epoch: 20 Loss: 0.002099
[03.04|15:31:28] validation_set Optimizer: 001 Epoch: 20 Loss: 0.014046
[03.04|15:31:35] training_set Optimizer: 002 Epoch: 30 Loss: 0.001470
[03.04|15:31:35] validation_set Optimizer: 002 Epoch: 30 Loss: 0.024544
[03.04|15:31:36] training_set Optimizer: 001 Epoch: 30 Loss: 0.001329
[03.04|15:31:36] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014969
[03.04|15:31:43] training_set Optimizer: 002 Epoch: 40 Loss: 0.001280
[03.04|15:31:43] validation_set Optimizer: 002 Epoch: 40 Loss: 0.012941
[03.04|15:31:43] training_set Optimizer: 001 Epoch: 40 Loss: 0.001330
[03.04|15:31:43] validation_set Optimizer: 001 Epoch: 40 Loss: 0.014847
[03.04|15:31:51] training_set Optimizer: 002 Epoch: 50 Loss: 0.001314
[03.04|15:31:51] validation_set Optimizer: 002 Epoch: 50 Loss: 0.020921
[03.04|15:31:51] training_set Optimizer: 001 Epoch: 50 Loss: 0.001110
[03.04|15:31:51] validation_set Optimizer: 001 Epoch: 50 Loss: 0.015230
[03.04|15:31:59] training_set Optimizer: 002 Epoch: 60 Loss: 0.001162
[03.04|15:31:59] validation_set Optimizer: 002 Epoch: 60 Loss: 0.011506
[03.04|15:31:59] training_set Optimizer: 001 Epoch: 60 Loss: 0.001125
[03.04|15:31:59] validation_set Optimizer: 001 Epoch: 60 Loss: 0.013790
[03.04|15:32:07] training_set Optimizer: 002 Epoch: 70 Loss: 0.001110
[03.04|15:32:07] validation_set Optimizer: 002 Epoch: 70 Loss: 0.012595
[03.04|15:32:07] training_set Optimizer: 001 Epoch: 70 Loss: 0.001135
[03.04|15:32:07] validation_set Optimizer: 001 Epoch: 70 Loss: 0.014817
[03.04|15:32:14] training_set Optimizer: 002 Epoch: 80 Loss: 0.001028
[03.04|15:32:14] validation_set Optimizer: 002 Epoch: 80 Loss: 0.022925
[03.04|15:32:14] training_set Optimizer: 001 Epoch: 80 Loss: 0.001065
[03.04|15:32:14] validation_set Optimizer: 001 Epoch: 80 Loss: 0.016624
[03.04|15:32:22] training_set Optimizer: 001 Epoch: 90 Loss: 0.001026
[03.04|15:32:22] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013000
[03.04|15:32:22] training_set Optimizer: 002 Epoch: 90 Loss: 0.001144
[03.04|15:32:22] validation_set Optimizer: 002 Epoch: 90 Loss: 0.011121
[03.04|15:32:30] training_set Optimizer: 001 Epoch: 100 Loss: 0.001631
[03.04|15:32:30] validation_set Optimizer: 001 Epoch: 100 Loss: 0.015989
[03.04|15:32:30] training_set Optimizer: 002 Epoch: 100 Loss: 0.000992
[03.04|15:32:30] validation_set Optimizer: 002 Epoch: 100 Loss: 0.021024
[03.04|15:32:38] training_set Optimizer: 002 Epoch: 110 Loss: 0.001360
[03.04|15:32:38] validation_set Optimizer: 002 Epoch: 110 Loss: 0.014653
[03.04|15:32:38] training_set Optimizer: 001 Epoch: 110 Loss: 0.001743
[03.04|15:32:38] validation_set Optimizer: 001 Epoch: 110 Loss: 0.015275
[03.04|15:32:47] Execution of optimizer_001.run finished with returncode 0
[03.04|15:32:47] Execution of optimizer_002.run finished with returncode 0
[03.04|15:32:47] JOB optimizer_001 FINISHED
[03.04|15:32:47] Starting optimizer_001.postrun()
[03.04|15:32:47] optimizer_001.postrun() finished
[03.04|15:32:47] JOB optimizer_002 FINISHED
[03.04|15:32:47] Starting optimizer_002.postrun()
[03.04|15:32:47] optimizer_002.postrun() finished
[03.04|15:32:48] JOB optimizer_002 SUCCESSFUL
[03.04|15:32:48] JOB optimizer_001 SUCCESSFUL
[03.04|15:32:48] PLAMS environment cleaned up successfully
[03.04|15:32:48] PLAMS run finished. Goodbye
[03.04|15:32:48] ParAMSResults
[03.04|15:32:48] Newly created parameter file/dir: step5_attempt2_training/results/optimization/optimizer_001/m3gnet
[03.04|15:32:48] Newly created parameter file/dir: step5_attempt2_training/results/optimization/optimizer_002/m3gnet
[03.04|15:32:48] Done!
[03.04|15:32:48] Deleting step5_attempt1_training
[03.04|15:32:48] ##########################
[03.04|15:32:48] ### Step 5 / Attempt 3 ###
[03.04|15:32:48] ##########################
[03.04|15:32:48] MD Steps: 2280 (cumulative: 3000)
[03.04|15:32:48] Current engine settings:
[03.04|15:32:48]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt2_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt2_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:32:48] Running step5_attempt3_simulation...
[03.04|15:33:50] Job step5_attempt3_simulation finished
[03.04|15:33:50] Deleting files that are no longer needed...
[03.04|15:33:50] Energy uncertainty for final frame of step5_attempt3_simulation: 0.0876 eV
[03.04|15:33:50] 0.0073 eV/atom
[03.04|15:33:50] Forces uncertainty for final frame of step5_attempt3_simulation: 1.1089 eV/angstrom
[03.04|15:33:51] Launching reference calculation
[03.04|15:34:07] Reference calculation finished!
[03.04|15:34:07] Adding results from step5_attempt3_reference_calc1 to training set
[03.04|15:34:07] Current # training set entries: 57
[03.04|15:34:07] Current # validation set entries: 21
[03.04|15:34:07] Storing data in step5_attempt3_reference_data
[03.04|15:34:07] Deleting step5_attempt2_reference_data
[03.04|15:34:07] Deleting step5_attempt3_reference_calc1
[03.04|15:34:07]
[03.04|15:34:07] Current (cumulative) timings:
[03.04|15:34:07] Time (s) Fraction
[03.04|15:34:07] Ref. calcs 548.46 0.143
[03.04|15:34:07] ML training 2464.71 0.641
[03.04|15:34:07] Simulations 832.63 0.217
[03.04|15:34:07]
[03.04|15:34:07]
[03.04|15:34:07]
[03.04|15:34:07] --- Begin summary ---
[03.04|15:34:07] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:34:07] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:34:07] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:34:07] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:34:07] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:34:07] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:34:07] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:34:07] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:34:07] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:34:07] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:34:07] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:34:07] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:34:07] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:34:07] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:34:07] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:34:07] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:34:07] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:34:07] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:34:07] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:34:07] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:34:07] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:34:07] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:34:07] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:34:07] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:34:07] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:34:07] --- End summary ---
[03.04|15:34:07]
[03.04|15:34:07] Running more reference calculations....
[03.04|15:34:07] Running reference calculations on frames [131, 173] from step5_attempt3_simulation/ams.rkf
[03.04|15:34:07] Calculating 2 frames in total
[03.04|15:34:07] Running step5_attempt3_reference_calc2
[03.04|15:34:22] Running step5_attempt3_reference_calc3
[03.04|15:34:38] Reference calculations finished!
[03.04|15:34:38] Adding results from step5_attempt3_reference_calc2 to training set
[03.04|15:34:38] Adding results from step5_attempt3_reference_calc3 to training set
[03.04|15:34:38] Current # training set entries: 59
[03.04|15:34:38] Current # validation set entries: 21
[03.04|15:34:38] Storing data in step5_attempt3_reference_data
[03.04|15:34:39] Deleting step5_attempt3_reference_calc2
[03.04|15:34:39] Deleting step5_attempt3_reference_calc3
[03.04|15:34:39] Launching reparametrization job: step5_attempt3_training
[03.04|15:34:43] JOB optimizer_001 STARTED
[03.04|15:34:43] JOB optimizer_002 STARTED
[03.04|15:34:43] Starting optimizer_001.prerun()
[03.04|15:34:43] Starting optimizer_002.prerun()
[03.04|15:34:43] optimizer_001.prerun() finished
[03.04|15:34:43] optimizer_002.prerun() finished
[03.04|15:34:43] JOB optimizer_002 RUNNING
[03.04|15:34:44] Executing optimizer_002.run
[03.04|15:34:44] Waiting for job optimizer_001 to finish
[03.04|15:34:44] JOB optimizer_001 RUNNING
[03.04|15:34:44] Executing optimizer_001.run
[03.04|15:36:02] training_set Optimizer: 002 Epoch: 0 Loss: 0.003054
[03.04|15:36:02] validation_set Optimizer: 002 Epoch: 0 Loss: 0.014244
[03.04|15:36:03] training_set Optimizer: 001 Epoch: 0 Loss: 0.004151
[03.04|15:36:03] validation_set Optimizer: 001 Epoch: 0 Loss: 0.030836
[03.04|15:36:11] training_set Optimizer: 002 Epoch: 10 Loss: 0.001008
[03.04|15:36:11] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009311
[03.04|15:36:11] training_set Optimizer: 001 Epoch: 10 Loss: 0.001240
[03.04|15:36:11] validation_set Optimizer: 001 Epoch: 10 Loss: 0.014351
[03.04|15:36:19] training_set Optimizer: 002 Epoch: 20 Loss: 0.001122
[03.04|15:36:19] validation_set Optimizer: 002 Epoch: 20 Loss: 0.009605
[03.04|15:36:19] training_set Optimizer: 001 Epoch: 20 Loss: 0.001233
[03.04|15:36:19] validation_set Optimizer: 001 Epoch: 20 Loss: 0.012944
[03.04|15:36:27] training_set Optimizer: 002 Epoch: 30 Loss: 0.001143
[03.04|15:36:27] validation_set Optimizer: 002 Epoch: 30 Loss: 0.011514
[03.04|15:36:27] training_set Optimizer: 001 Epoch: 30 Loss: 0.001245
[03.04|15:36:27] validation_set Optimizer: 001 Epoch: 30 Loss: 0.018660
[03.04|15:36:35] training_set Optimizer: 002 Epoch: 40 Loss: 0.000913
[03.04|15:36:35] validation_set Optimizer: 002 Epoch: 40 Loss: 0.013163
[03.04|15:36:36] training_set Optimizer: 001 Epoch: 40 Loss: 0.001067
[03.04|15:36:36] validation_set Optimizer: 001 Epoch: 40 Loss: 0.016898
[03.04|15:36:43] training_set Optimizer: 002 Epoch: 50 Loss: 0.000910
[03.04|15:36:43] validation_set Optimizer: 002 Epoch: 50 Loss: 0.012371
[03.04|15:36:44] training_set Optimizer: 001 Epoch: 50 Loss: 0.000998
[03.04|15:36:44] validation_set Optimizer: 001 Epoch: 50 Loss: 0.014335
[03.04|15:36:52] training_set Optimizer: 002 Epoch: 60 Loss: 0.001036
[03.04|15:36:52] validation_set Optimizer: 002 Epoch: 60 Loss: 0.014822
[03.04|15:36:52] training_set Optimizer: 001 Epoch: 60 Loss: 0.001105
[03.04|15:36:52] validation_set Optimizer: 001 Epoch: 60 Loss: 0.018874
[03.04|15:37:00] training_set Optimizer: 002 Epoch: 70 Loss: 0.000821
[03.04|15:37:00] validation_set Optimizer: 002 Epoch: 70 Loss: 0.024263
[03.04|15:37:01] training_set Optimizer: 001 Epoch: 70 Loss: 0.001205
[03.04|15:37:01] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011757
[03.04|15:37:08] training_set Optimizer: 002 Epoch: 80 Loss: 0.000755
[03.04|15:37:08] validation_set Optimizer: 002 Epoch: 80 Loss: 0.008622
[03.04|15:37:09] training_set Optimizer: 001 Epoch: 80 Loss: 0.001022
[03.04|15:37:09] validation_set Optimizer: 001 Epoch: 80 Loss: 0.013476
[03.04|15:37:17] training_set Optimizer: 002 Epoch: 90 Loss: 0.001131
[03.04|15:37:17] validation_set Optimizer: 002 Epoch: 90 Loss: 0.011561
[03.04|15:37:17] training_set Optimizer: 001 Epoch: 90 Loss: 0.001038
[03.04|15:37:17] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013232
[03.04|15:37:25] training_set Optimizer: 002 Epoch: 100 Loss: 0.001123
[03.04|15:37:25] validation_set Optimizer: 002 Epoch: 100 Loss: 0.015504
[03.04|15:37:25] training_set Optimizer: 001 Epoch: 100 Loss: 0.000971
[03.04|15:37:25] validation_set Optimizer: 001 Epoch: 100 Loss: 0.015685
[03.04|15:37:34] training_set Optimizer: 002 Epoch: 110 Loss: 0.000793
[03.04|15:37:34] validation_set Optimizer: 002 Epoch: 110 Loss: 0.012756
[03.04|15:37:34] training_set Optimizer: 001 Epoch: 110 Loss: 0.000903
[03.04|15:37:34] validation_set Optimizer: 001 Epoch: 110 Loss: 0.016791
[03.04|15:37:43] Execution of optimizer_002.run finished with returncode 0
[03.04|15:37:44] JOB optimizer_002 FINISHED
[03.04|15:37:44] Starting optimizer_002.postrun()
[03.04|15:37:44] optimizer_002.postrun() finished
[03.04|15:37:44] Execution of optimizer_001.run finished with returncode 0
[03.04|15:37:44] JOB optimizer_001 FINISHED
[03.04|15:37:44] Starting optimizer_001.postrun()
[03.04|15:37:44] optimizer_001.postrun() finished
[03.04|15:37:44] JOB optimizer_002 SUCCESSFUL
[03.04|15:37:44] JOB optimizer_001 SUCCESSFUL
[03.04|15:37:44] PLAMS environment cleaned up successfully
[03.04|15:37:44] PLAMS run finished. Goodbye
[03.04|15:37:45] ParAMSResults
[03.04|15:37:45] Newly created parameter file/dir: step5_attempt3_training/results/optimization/optimizer_001/m3gnet
[03.04|15:37:45] Newly created parameter file/dir: step5_attempt3_training/results/optimization/optimizer_002/m3gnet
[03.04|15:37:45] Done!
[03.04|15:37:45] Deleting step5_attempt2_training
[03.04|15:37:45] ##########################
[03.04|15:37:45] ### Step 5 / Attempt 4 ###
[03.04|15:37:45] ##########################
[03.04|15:37:45] MD Steps: 2280 (cumulative: 3000)
[03.04|15:37:45] Current engine settings:
[03.04|15:37:45]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt3_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt3_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:37:45] Running step5_attempt4_simulation...
[03.04|15:38:37] Job step5_attempt4_simulation finished
[03.04|15:38:37] Deleting files that are no longer needed...
[03.04|15:38:37] Energy uncertainty for final frame of step5_attempt4_simulation: 0.0209 eV
[03.04|15:38:37] 0.0017 eV/atom
[03.04|15:38:37] Forces uncertainty for final frame of step5_attempt4_simulation: 1.0169 eV/angstrom
[03.04|15:38:38] Launching reference calculation
[03.04|15:38:52] Reference calculation finished!
[03.04|15:38:53] Adding results from step5_attempt4_reference_calc1 to training set
[03.04|15:38:53] Current # training set entries: 60
[03.04|15:38:53] Current # validation set entries: 21
[03.04|15:38:53] Storing data in step5_attempt4_reference_data
[03.04|15:38:53] Deleting step5_attempt3_reference_data
[03.04|15:38:53] Deleting step5_attempt4_reference_calc1
[03.04|15:38:53]
[03.04|15:38:53] Current (cumulative) timings:
[03.04|15:38:53] Time (s) Fraction
[03.04|15:38:53] Ref. calcs 594.09 0.144
[03.04|15:38:53] ML training 2650.98 0.642
[03.04|15:38:53] Simulations 884.85 0.214
[03.04|15:38:53]
[03.04|15:38:53]
[03.04|15:38:53]
[03.04|15:38:53] --- Begin summary ---
[03.04|15:38:53] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:38:53] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:38:53] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:38:53] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:38:53] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:38:53] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:38:53] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:38:53] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:38:53] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:38:53] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:38:53] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:38:53] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:38:53] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:38:53] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:38:53] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:38:53] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:38:53] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:38:53] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:38:53] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:38:53] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:38:53] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:38:53] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:38:53] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:38:53] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:38:53] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:38:53] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|15:38:53] --- End summary ---
[03.04|15:38:53]
[03.04|15:38:53] Running more reference calculations....
[03.04|15:38:53] Running reference calculations on frames [134, 156] from step5_attempt4_simulation/ams.rkf
[03.04|15:38:53] Calculating 2 frames in total
[03.04|15:38:53] Running step5_attempt4_reference_calc2
[03.04|15:39:09] Running step5_attempt4_reference_calc3
[03.04|15:39:25] Reference calculations finished!
[03.04|15:39:26] Adding results from step5_attempt4_reference_calc2 to training set
[03.04|15:39:26] Adding results from step5_attempt4_reference_calc3 to training set
[03.04|15:39:26] Current # training set entries: 62
[03.04|15:39:26] Current # validation set entries: 21
[03.04|15:39:26] Storing data in step5_attempt4_reference_data
[03.04|15:39:26] Deleting step5_attempt4_reference_calc2
[03.04|15:39:26] Deleting step5_attempt4_reference_calc3
[03.04|15:39:26] Launching reparametrization job: step5_attempt4_training
[03.04|15:39:31] JOB optimizer_001 STARTED
[03.04|15:39:31] JOB optimizer_002 STARTED
[03.04|15:39:31] Starting optimizer_001.prerun()
[03.04|15:39:31] optimizer_001.prerun() finished
[03.04|15:39:31] Starting optimizer_002.prerun()
[03.04|15:39:31] optimizer_002.prerun() finished
[03.04|15:39:31] JOB optimizer_002 RUNNING
[03.04|15:39:31] Executing optimizer_002.run
[03.04|15:39:31] JOB optimizer_001 RUNNING
[03.04|15:39:31] Waiting for job optimizer_001 to finish
[03.04|15:39:31] Executing optimizer_001.run
[03.04|15:40:48] training_set Optimizer: 001 Epoch: 0 Loss: 0.002326
[03.04|15:40:48] validation_set Optimizer: 001 Epoch: 0 Loss: 0.018200
[03.04|15:40:55] training_set Optimizer: 001 Epoch: 10 Loss: 0.001108
[03.04|15:40:55] validation_set Optimizer: 001 Epoch: 10 Loss: 0.027353
[03.04|15:41:04] training_set Optimizer: 001 Epoch: 20 Loss: 0.000977
[03.04|15:41:04] validation_set Optimizer: 001 Epoch: 20 Loss: 0.016500
[03.04|15:41:08] training_set Optimizer: 002 Epoch: 0 Loss: 0.002724
[03.04|15:41:08] validation_set Optimizer: 002 Epoch: 0 Loss: 0.028128
[03.04|15:41:13] training_set Optimizer: 001 Epoch: 30 Loss: 0.000880
[03.04|15:41:13] validation_set Optimizer: 001 Epoch: 30 Loss: 0.012235
[03.04|15:41:18] training_set Optimizer: 002 Epoch: 10 Loss: 0.000933
[03.04|15:41:18] validation_set Optimizer: 002 Epoch: 10 Loss: 0.010950
[03.04|15:41:22] training_set Optimizer: 001 Epoch: 40 Loss: 0.000919
[03.04|15:41:22] validation_set Optimizer: 001 Epoch: 40 Loss: 0.012978
[03.04|15:41:27] training_set Optimizer: 002 Epoch: 20 Loss: 0.001180
[03.04|15:41:27] validation_set Optimizer: 002 Epoch: 20 Loss: 0.011394
[03.04|15:41:31] training_set Optimizer: 001 Epoch: 50 Loss: 0.001051
[03.04|15:41:31] validation_set Optimizer: 001 Epoch: 50 Loss: 0.013153
[03.04|15:41:35] training_set Optimizer: 002 Epoch: 30 Loss: 0.000920
[03.04|15:41:35] validation_set Optimizer: 002 Epoch: 30 Loss: 0.016710
[03.04|15:41:39] training_set Optimizer: 001 Epoch: 60 Loss: 0.000870
[03.04|15:41:39] validation_set Optimizer: 001 Epoch: 60 Loss: 0.016453
[03.04|15:41:43] training_set Optimizer: 002 Epoch: 40 Loss: 0.000882
[03.04|15:41:43] validation_set Optimizer: 002 Epoch: 40 Loss: 0.009882
[03.04|15:41:47] training_set Optimizer: 001 Epoch: 70 Loss: 0.000812
[03.04|15:41:47] validation_set Optimizer: 001 Epoch: 70 Loss: 0.013742
[03.04|15:41:51] training_set Optimizer: 002 Epoch: 50 Loss: 0.000895
[03.04|15:41:51] validation_set Optimizer: 002 Epoch: 50 Loss: 0.008958
[03.04|15:41:55] training_set Optimizer: 001 Epoch: 80 Loss: 0.000787
[03.04|15:41:55] validation_set Optimizer: 001 Epoch: 80 Loss: 0.011840
[03.04|15:42:00] training_set Optimizer: 002 Epoch: 60 Loss: 0.000731
[03.04|15:42:00] validation_set Optimizer: 002 Epoch: 60 Loss: 0.018359
[03.04|15:42:04] training_set Optimizer: 001 Epoch: 90 Loss: 0.000992
[03.04|15:42:04] validation_set Optimizer: 001 Epoch: 90 Loss: 0.020596
[03.04|15:42:08] training_set Optimizer: 002 Epoch: 70 Loss: 0.000787
[03.04|15:42:08] validation_set Optimizer: 002 Epoch: 70 Loss: 0.011180
[03.04|15:42:12] training_set Optimizer: 001 Epoch: 100 Loss: 0.000948
[03.04|15:42:12] validation_set Optimizer: 001 Epoch: 100 Loss: 0.018422
[03.04|15:42:16] training_set Optimizer: 002 Epoch: 80 Loss: 0.000743
[03.04|15:42:16] validation_set Optimizer: 002 Epoch: 80 Loss: 0.010799
[03.04|15:42:20] training_set Optimizer: 001 Epoch: 110 Loss: 0.000744
[03.04|15:42:20] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011777
[03.04|15:42:24] training_set Optimizer: 002 Epoch: 90 Loss: 0.000984
[03.04|15:42:24] validation_set Optimizer: 002 Epoch: 90 Loss: 0.012528
[03.04|15:42:30] Execution of optimizer_001.run finished with returncode 0
[03.04|15:42:30] JOB optimizer_001 FINISHED
[03.04|15:42:30] Starting optimizer_001.postrun()
[03.04|15:42:30] optimizer_001.postrun() finished
[03.04|15:42:30] JOB optimizer_001 SUCCESSFUL
[03.04|15:42:30] Waiting for job optimizer_002 to finish
[03.04|15:42:32] training_set Optimizer: 002 Epoch: 100 Loss: 0.000673
[03.04|15:42:32] validation_set Optimizer: 002 Epoch: 100 Loss: 0.012026
[03.04|15:42:38] training_set Optimizer: 002 Epoch: 110 Loss: 0.000731
[03.04|15:42:38] validation_set Optimizer: 002 Epoch: 110 Loss: 0.016130
[03.04|15:42:47] Execution of optimizer_002.run finished with returncode 0
[03.04|15:42:47] JOB optimizer_002 FINISHED
[03.04|15:42:47] Starting optimizer_002.postrun()
[03.04|15:42:47] optimizer_002.postrun() finished
[03.04|15:42:47] JOB optimizer_002 SUCCESSFUL
[03.04|15:42:47] PLAMS environment cleaned up successfully
[03.04|15:42:47] PLAMS run finished. Goodbye
[03.04|15:42:48] ParAMSResults
[03.04|15:42:48] Newly created parameter file/dir: step5_attempt4_training/results/optimization/optimizer_001/m3gnet
[03.04|15:42:48] Newly created parameter file/dir: step5_attempt4_training/results/optimization/optimizer_002/m3gnet
[03.04|15:42:48] Done!
[03.04|15:42:48] Deleting step5_attempt3_training
[03.04|15:42:48] ##########################
[03.04|15:42:48] ### Step 5 / Attempt 5 ###
[03.04|15:42:48] ##########################
[03.04|15:42:48] MD Steps: 2280 (cumulative: 3000)
[03.04|15:42:48] Current engine settings:
[03.04|15:42:48]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt4_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt4_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:42:48] Running step5_attempt5_simulation...
[03.04|15:43:50] Job step5_attempt5_simulation finished
[03.04|15:43:50] Deleting files that are no longer needed...
[03.04|15:43:50] Energy uncertainty for final frame of step5_attempt5_simulation: 0.1188 eV
[03.04|15:43:50] 0.0099 eV/atom
[03.04|15:43:50] Forces uncertainty for final frame of step5_attempt5_simulation: 1.0401 eV/angstrom
[03.04|15:43:50] Launching reference calculation
[03.04|15:44:04] Reference calculation finished!
[03.04|15:44:05] Adding results from step5_attempt5_reference_calc1 to training set
[03.04|15:44:05] Current # training set entries: 63
[03.04|15:44:05] Current # validation set entries: 21
[03.04|15:44:05] Storing data in step5_attempt5_reference_data
[03.04|15:44:05] Deleting step5_attempt4_reference_data
[03.04|15:44:05] Deleting step5_attempt5_reference_calc1
[03.04|15:44:05]
[03.04|15:44:05] Current (cumulative) timings:
[03.04|15:44:05] Time (s) Fraction
[03.04|15:44:05] Ref. calcs 640.72 0.144
[03.04|15:44:05] ML training 2853.15 0.643
[03.04|15:44:05] Simulations 946.53 0.213
[03.04|15:44:05]
[03.04|15:44:05]
[03.04|15:44:05]
[03.04|15:44:05] --- Begin summary ---
[03.04|15:44:05] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:44:05] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:44:05] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:44:05] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:44:05] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:44:05] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:44:05] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:44:05] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:44:05] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:44:05] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:44:05] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:44:05] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:44:05] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:44:05] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:44:05] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:44:05] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:44:05] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:44:05] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:44:05] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:44:05] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:44:05] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:44:05] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:44:05] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:44:05] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:44:05] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:44:05] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|15:44:05] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|15:44:05] --- End summary ---
[03.04|15:44:05]
[03.04|15:44:05] Running more reference calculations....
[03.04|15:44:05] Running reference calculations on frames [168, 208] from step5_attempt5_simulation/ams.rkf
[03.04|15:44:05] Calculating 2 frames in total
[03.04|15:44:05] Running step5_attempt5_reference_calc2
[03.04|15:44:19] Running step5_attempt5_reference_calc3
[03.04|15:44:35] Reference calculations finished!
[03.04|15:44:35] Adding results from step5_attempt5_reference_calc2 to training set
[03.04|15:44:35] Adding results from step5_attempt5_reference_calc3 to training set
[03.04|15:44:35] Current # training set entries: 65
[03.04|15:44:35] Current # validation set entries: 21
[03.04|15:44:35] Storing data in step5_attempt5_reference_data
[03.04|15:44:36] Deleting step5_attempt5_reference_calc2
[03.04|15:44:36] Deleting step5_attempt5_reference_calc3
[03.04|15:44:36] Launching reparametrization job: step5_attempt5_training
[03.04|15:44:40] JOB optimizer_001 STARTED
[03.04|15:44:40] JOB optimizer_002 STARTED
[03.04|15:44:40] Starting optimizer_001.prerun()
[03.04|15:44:40] optimizer_001.prerun() finished
[03.04|15:44:40] Starting optimizer_002.prerun()
[03.04|15:44:40] optimizer_002.prerun() finished
[03.04|15:44:40] JOB optimizer_001 RUNNING
[03.04|15:44:40] JOB optimizer_002 RUNNING
[03.04|15:44:40] Executing optimizer_001.run
[03.04|15:44:40] Executing optimizer_002.run
[03.04|15:44:40] Waiting for job optimizer_001 to finish
[03.04|15:45:36] training_set Optimizer: 002 Epoch: 0 Loss: 0.004244
[03.04|15:45:36] validation_set Optimizer: 002 Epoch: 0 Loss: 0.040810
[03.04|15:45:36] training_set Optimizer: 001 Epoch: 0 Loss: 0.002756
[03.04|15:45:36] validation_set Optimizer: 001 Epoch: 0 Loss: 0.016590
[03.04|15:45:44] training_set Optimizer: 002 Epoch: 10 Loss: 0.000874
[03.04|15:45:44] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009254
[03.04|15:45:45] training_set Optimizer: 001 Epoch: 10 Loss: 0.001344
[03.04|15:45:45] validation_set Optimizer: 001 Epoch: 10 Loss: 0.013699
[03.04|15:45:53] training_set Optimizer: 002 Epoch: 20 Loss: 0.000958
[03.04|15:45:53] validation_set Optimizer: 002 Epoch: 20 Loss: 0.010412
[03.04|15:45:54] training_set Optimizer: 001 Epoch: 20 Loss: 0.001123
[03.04|15:45:54] validation_set Optimizer: 001 Epoch: 20 Loss: 0.019156
[03.04|15:46:02] training_set Optimizer: 002 Epoch: 30 Loss: 0.000742
[03.04|15:46:02] validation_set Optimizer: 002 Epoch: 30 Loss: 0.008154
[03.04|15:46:03] training_set Optimizer: 001 Epoch: 30 Loss: 0.001026
[03.04|15:46:03] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014026
[03.04|15:46:11] training_set Optimizer: 002 Epoch: 40 Loss: 0.000800
[03.04|15:46:11] validation_set Optimizer: 002 Epoch: 40 Loss: 0.010119
[03.04|15:46:11] training_set Optimizer: 001 Epoch: 40 Loss: 0.000878
[03.04|15:46:11] validation_set Optimizer: 001 Epoch: 40 Loss: 0.014771
[03.04|15:46:19] training_set Optimizer: 002 Epoch: 50 Loss: 0.000706
[03.04|15:46:19] validation_set Optimizer: 002 Epoch: 50 Loss: 0.012978
[03.04|15:46:20] training_set Optimizer: 001 Epoch: 50 Loss: 0.000859
[03.04|15:46:20] validation_set Optimizer: 001 Epoch: 50 Loss: 0.017368
[03.04|15:46:27] training_set Optimizer: 002 Epoch: 60 Loss: 0.000941
[03.04|15:46:27] validation_set Optimizer: 002 Epoch: 60 Loss: 0.010766
[03.04|15:46:28] training_set Optimizer: 001 Epoch: 60 Loss: 0.000828
[03.04|15:46:28] validation_set Optimizer: 001 Epoch: 60 Loss: 0.015984
[03.04|15:46:36] training_set Optimizer: 002 Epoch: 70 Loss: 0.000735
[03.04|15:46:36] validation_set Optimizer: 002 Epoch: 70 Loss: 0.008517
[03.04|15:46:37] training_set Optimizer: 001 Epoch: 70 Loss: 0.000811
[03.04|15:46:37] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011893
[03.04|15:46:44] training_set Optimizer: 002 Epoch: 80 Loss: 0.000757
[03.04|15:46:44] validation_set Optimizer: 002 Epoch: 80 Loss: 0.009464
[03.04|15:46:45] training_set Optimizer: 001 Epoch: 80 Loss: 0.000775
[03.04|15:46:45] validation_set Optimizer: 001 Epoch: 80 Loss: 0.011820
[03.04|15:46:53] training_set Optimizer: 002 Epoch: 90 Loss: 0.000633
[03.04|15:46:53] validation_set Optimizer: 002 Epoch: 90 Loss: 0.014511
[03.04|15:46:54] training_set Optimizer: 001 Epoch: 90 Loss: 0.001077
[03.04|15:46:54] validation_set Optimizer: 001 Epoch: 90 Loss: 0.012484
[03.04|15:47:01] training_set Optimizer: 002 Epoch: 100 Loss: 0.000594
[03.04|15:47:01] validation_set Optimizer: 002 Epoch: 100 Loss: 0.016196
[03.04|15:47:02] training_set Optimizer: 001 Epoch: 100 Loss: 0.000663
[03.04|15:47:02] validation_set Optimizer: 001 Epoch: 100 Loss: 0.012793
[03.04|15:47:11] training_set Optimizer: 002 Epoch: 110 Loss: 0.000592
[03.04|15:47:11] validation_set Optimizer: 002 Epoch: 110 Loss: 0.009356
[03.04|15:47:12] training_set Optimizer: 001 Epoch: 110 Loss: 0.000648
[03.04|15:47:12] validation_set Optimizer: 001 Epoch: 110 Loss: 0.010890
[03.04|15:47:21] Execution of optimizer_002.run finished with returncode 0
[03.04|15:47:21] JOB optimizer_002 FINISHED
[03.04|15:47:21] Starting optimizer_002.postrun()
[03.04|15:47:21] optimizer_002.postrun() finished
[03.04|15:47:21] JOB optimizer_002 SUCCESSFUL
[03.04|15:47:22] Execution of optimizer_001.run finished with returncode 0
[03.04|15:47:22] JOB optimizer_001 FINISHED
[03.04|15:47:22] Starting optimizer_001.postrun()
[03.04|15:47:22] optimizer_001.postrun() finished
[03.04|15:47:22] JOB optimizer_001 SUCCESSFUL
[03.04|15:47:22] PLAMS environment cleaned up successfully
[03.04|15:47:22] PLAMS run finished. Goodbye
[03.04|15:47:23] ParAMSResults
[03.04|15:47:23] Newly created parameter file/dir: step5_attempt5_training/results/optimization/optimizer_001/m3gnet
[03.04|15:47:23] Newly created parameter file/dir: step5_attempt5_training/results/optimization/optimizer_002/m3gnet
[03.04|15:47:23] Done!
[03.04|15:47:23] Deleting step5_attempt4_training
[03.04|15:47:23] ##########################
[03.04|15:47:23] ### Step 5 / Attempt 6 ###
[03.04|15:47:23] ##########################
[03.04|15:47:23] MD Steps: 2280 (cumulative: 3000)
[03.04|15:47:23] Current engine settings:
[03.04|15:47:23]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt5_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt5_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:47:23] Running step5_attempt6_simulation...
[03.04|15:48:32] Job step5_attempt6_simulation finished
[03.04|15:48:32] Deleting files that are no longer needed...
[03.04|15:48:32] Energy uncertainty for final frame of step5_attempt6_simulation: 0.0966 eV
[03.04|15:48:32] 0.0080 eV/atom
[03.04|15:48:32] Forces uncertainty for final frame of step5_attempt6_simulation: 1.0176 eV/angstrom
[03.04|15:48:33] Launching reference calculation
[03.04|15:48:50] Reference calculation finished!
[03.04|15:48:50] Adding results from step5_attempt6_reference_calc1 to training set
[03.04|15:48:50] Current # training set entries: 66
[03.04|15:48:50] Current # validation set entries: 21
[03.04|15:48:50] Storing data in step5_attempt6_reference_data
[03.04|15:48:51] Deleting step5_attempt5_reference_data
[03.04|15:48:51] Deleting step5_attempt6_reference_calc1
[03.04|15:48:51]
[03.04|15:48:51] Current (cumulative) timings:
[03.04|15:48:51] Time (s) Fraction
[03.04|15:48:51] Ref. calcs 688.20 0.146
[03.04|15:48:51] ML training 3020.57 0.639
[03.04|15:48:51] Simulations 1015.77 0.215
[03.04|15:48:51]
[03.04|15:48:51]
[03.04|15:48:51]
[03.04|15:48:51] --- Begin summary ---
[03.04|15:48:51] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:48:51] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:48:51] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:48:51] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:48:51] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:48:51] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:48:51] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:48:51] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:48:51] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:48:51] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:48:51] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:48:51] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:48:51] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:48:51] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:48:51] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:48:51] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:48:51] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:48:51] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:48:51] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:48:51] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:48:51] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:48:51] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:48:51] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:48:51] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:48:51] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:48:51] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|15:48:51] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|15:48:51] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|15:48:51] --- End summary ---
[03.04|15:48:51]
[03.04|15:48:51] Running more reference calculations....
[03.04|15:48:51] Running reference calculations on frames [174, 216] from step5_attempt6_simulation/ams.rkf
[03.04|15:48:51] Calculating 2 frames in total
[03.04|15:48:51] Running step5_attempt6_reference_calc2
[03.04|15:49:07] Running step5_attempt6_reference_calc3
[03.04|15:49:23] Reference calculations finished!
[03.04|15:49:23] Adding results from step5_attempt6_reference_calc2 to training set
[03.04|15:49:23] Adding results from step5_attempt6_reference_calc3 to training set
[03.04|15:49:23] Current # training set entries: 68
[03.04|15:49:23] Current # validation set entries: 21
[03.04|15:49:23] Storing data in step5_attempt6_reference_data
[03.04|15:49:24] Deleting step5_attempt6_reference_calc2
[03.04|15:49:24] Deleting step5_attempt6_reference_calc3
[03.04|15:49:24] Launching reparametrization job: step5_attempt6_training
[03.04|15:49:28] JOB optimizer_001 STARTED
[03.04|15:49:28] JOB optimizer_002 STARTED
[03.04|15:49:28] Starting optimizer_001.prerun()
[03.04|15:49:28] optimizer_001.prerun() finished
[03.04|15:49:28] Starting optimizer_002.prerun()
[03.04|15:49:28] optimizer_002.prerun() finished
[03.04|15:49:28] JOB optimizer_002 RUNNING
[03.04|15:49:28] Executing optimizer_002.run
[03.04|15:49:28] JOB optimizer_001 RUNNING
[03.04|15:49:28] Executing optimizer_001.run
[03.04|15:49:28] Waiting for job optimizer_001 to finish
[03.04|15:50:44] training_set Optimizer: 001 Epoch: 0 Loss: 0.003553
[03.04|15:50:44] validation_set Optimizer: 001 Epoch: 0 Loss: 0.018397
[03.04|15:50:45] training_set Optimizer: 002 Epoch: 0 Loss: 0.002872
[03.04|15:50:45] validation_set Optimizer: 002 Epoch: 0 Loss: 0.010041
[03.04|15:50:52] training_set Optimizer: 001 Epoch: 10 Loss: 0.000860
[03.04|15:50:52] validation_set Optimizer: 001 Epoch: 10 Loss: 0.011172
[03.04|15:50:54] training_set Optimizer: 002 Epoch: 10 Loss: 0.000894
[03.04|15:50:54] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009352
[03.04|15:51:01] training_set Optimizer: 001 Epoch: 20 Loss: 0.000915
[03.04|15:51:01] validation_set Optimizer: 001 Epoch: 20 Loss: 0.013318
[03.04|15:51:03] training_set Optimizer: 002 Epoch: 20 Loss: 0.000903
[03.04|15:51:03] validation_set Optimizer: 002 Epoch: 20 Loss: 0.007899
[03.04|15:51:10] training_set Optimizer: 001 Epoch: 30 Loss: 0.000869
[03.04|15:51:10] validation_set Optimizer: 001 Epoch: 30 Loss: 0.025163
[03.04|15:51:12] training_set Optimizer: 002 Epoch: 30 Loss: 0.000827
[03.04|15:51:12] validation_set Optimizer: 002 Epoch: 30 Loss: 0.008616
[03.04|15:51:19] training_set Optimizer: 001 Epoch: 40 Loss: 0.000732
[03.04|15:51:19] validation_set Optimizer: 001 Epoch: 40 Loss: 0.013386
[03.04|15:51:21] training_set Optimizer: 002 Epoch: 40 Loss: 0.000671
[03.04|15:51:21] validation_set Optimizer: 002 Epoch: 40 Loss: 0.008194
[03.04|15:51:27] training_set Optimizer: 001 Epoch: 50 Loss: 0.000727
[03.04|15:51:27] validation_set Optimizer: 001 Epoch: 50 Loss: 0.012364
[03.04|15:51:30] training_set Optimizer: 002 Epoch: 50 Loss: 0.000943
[03.04|15:51:30] validation_set Optimizer: 002 Epoch: 50 Loss: 0.015386
[03.04|15:51:36] training_set Optimizer: 001 Epoch: 60 Loss: 0.000639
[03.04|15:51:36] validation_set Optimizer: 001 Epoch: 60 Loss: 0.021048
[03.04|15:51:39] training_set Optimizer: 002 Epoch: 60 Loss: 0.000667
[03.04|15:51:39] validation_set Optimizer: 002 Epoch: 60 Loss: 0.013812
[03.04|15:51:45] training_set Optimizer: 001 Epoch: 70 Loss: 0.000682
[03.04|15:51:45] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011323
[03.04|15:51:47] training_set Optimizer: 002 Epoch: 70 Loss: 0.000750
[03.04|15:51:47] validation_set Optimizer: 002 Epoch: 70 Loss: 0.007690
[03.04|15:51:54] training_set Optimizer: 001 Epoch: 80 Loss: 0.000831
[03.04|15:51:54] validation_set Optimizer: 001 Epoch: 80 Loss: 0.015933
[03.04|15:51:56] training_set Optimizer: 002 Epoch: 80 Loss: 0.000612
[03.04|15:51:56] validation_set Optimizer: 002 Epoch: 80 Loss: 0.010535
[03.04|15:52:02] training_set Optimizer: 001 Epoch: 90 Loss: 0.000972
[03.04|15:52:02] validation_set Optimizer: 001 Epoch: 90 Loss: 0.012469
[03.04|15:52:05] training_set Optimizer: 002 Epoch: 90 Loss: 0.000944
[03.04|15:52:05] validation_set Optimizer: 002 Epoch: 90 Loss: 0.009882
[03.04|15:52:12] training_set Optimizer: 001 Epoch: 100 Loss: 0.000825
[03.04|15:52:12] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013461
[03.04|15:52:14] training_set Optimizer: 002 Epoch: 100 Loss: 0.000703
[03.04|15:52:14] validation_set Optimizer: 002 Epoch: 100 Loss: 0.008029
[03.04|15:52:20] training_set Optimizer: 001 Epoch: 110 Loss: 0.000590
[03.04|15:52:20] validation_set Optimizer: 001 Epoch: 110 Loss: 0.010633
[03.04|15:52:23] training_set Optimizer: 002 Epoch: 110 Loss: 0.000746
[03.04|15:52:23] validation_set Optimizer: 002 Epoch: 110 Loss: 0.009754
[03.04|15:52:31] Execution of optimizer_001.run finished with returncode 0
[03.04|15:52:31] JOB optimizer_001 FINISHED
[03.04|15:52:31] Starting optimizer_001.postrun()
[03.04|15:52:31] optimizer_001.postrun() finished
[03.04|15:52:31] JOB optimizer_001 SUCCESSFUL
[03.04|15:52:31] Waiting for job optimizer_002 to finish
[03.04|15:52:33] Execution of optimizer_002.run finished with returncode 0
[03.04|15:52:33] JOB optimizer_002 FINISHED
[03.04|15:52:33] Starting optimizer_002.postrun()
[03.04|15:52:33] optimizer_002.postrun() finished
[03.04|15:52:34] JOB optimizer_002 SUCCESSFUL
[03.04|15:52:34] PLAMS environment cleaned up successfully
[03.04|15:52:34] PLAMS run finished. Goodbye
[03.04|15:52:34] ParAMSResults
[03.04|15:52:34] Newly created parameter file/dir: step5_attempt6_training/results/optimization/optimizer_001/m3gnet
[03.04|15:52:34] Newly created parameter file/dir: step5_attempt6_training/results/optimization/optimizer_002/m3gnet
[03.04|15:52:34] Done!
[03.04|15:52:34] Deleting step5_attempt5_training
[03.04|15:52:34] ##########################
[03.04|15:52:34] ### Step 5 / Attempt 7 ###
[03.04|15:52:34] ##########################
[03.04|15:52:34] MD Steps: 2280 (cumulative: 3000)
[03.04|15:52:34] Current engine settings:
[03.04|15:52:34]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt6_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt6_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:52:34] Running step5_attempt7_simulation...
[03.04|15:53:37] Job step5_attempt7_simulation finished
[03.04|15:53:37] Deleting files that are no longer needed...
[03.04|15:53:37] Energy uncertainty for final frame of step5_attempt7_simulation: 0.0020 eV
[03.04|15:53:37] 0.0002 eV/atom
[03.04|15:53:37] Forces uncertainty for final frame of step5_attempt7_simulation: 1.0315 eV/angstrom
[03.04|15:53:37] Launching reference calculation
[03.04|15:53:51] Reference calculation finished!
[03.04|15:53:52] Adding results from step5_attempt7_reference_calc1 to training set
[03.04|15:53:52] Current # training set entries: 69
[03.04|15:53:52] Current # validation set entries: 21
[03.04|15:53:52] Storing data in step5_attempt7_reference_data
[03.04|15:53:52] Deleting step5_attempt6_reference_data
[03.04|15:53:52] Deleting step5_attempt7_reference_calc1
[03.04|15:53:52]
[03.04|15:53:52] Current (cumulative) timings:
[03.04|15:53:52] Time (s) Fraction
[03.04|15:53:52] Ref. calcs 734.38 0.146
[03.04|15:53:52] ML training 3211.26 0.639
[03.04|15:53:52] Simulations 1078.46 0.215
[03.04|15:53:52]
[03.04|15:53:52]
[03.04|15:53:52]
[03.04|15:53:52] --- Begin summary ---
[03.04|15:53:52] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:53:52] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:53:52] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:53:52] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:53:52] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:53:52] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:53:52] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:53:52] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:53:52] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:53:52] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:53:52] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:53:52] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:53:52] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:53:52] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:53:52] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:53:52] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:53:52] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:53:52] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:53:52] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:53:52] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:53:52] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:53:52] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:53:52] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:53:52] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:53:52] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:53:52] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|15:53:52] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|15:53:52] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|15:53:52] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|15:53:52] --- End summary ---
[03.04|15:53:52]
[03.04|15:53:52] Running more reference calculations....
[03.04|15:53:52] Running reference calculations on frames [130, 171] from step5_attempt7_simulation/ams.rkf
[03.04|15:53:52] Calculating 2 frames in total
[03.04|15:53:52] Running step5_attempt7_reference_calc2
[03.04|15:54:06] Running step5_attempt7_reference_calc3
[03.04|15:54:20] Reference calculations finished!
[03.04|15:54:20] Adding results from step5_attempt7_reference_calc2 to training set
[03.04|15:54:20] Adding results from step5_attempt7_reference_calc3 to training set
[03.04|15:54:20] Current # training set entries: 71
[03.04|15:54:20] Current # validation set entries: 21
[03.04|15:54:20] Storing data in step5_attempt7_reference_data
[03.04|15:54:21] Deleting step5_attempt7_reference_calc2
[03.04|15:54:21] Deleting step5_attempt7_reference_calc3
[03.04|15:54:21] Launching reparametrization job: step5_attempt7_training
[03.04|15:54:25] JOB optimizer_001 STARTED
[03.04|15:54:25] JOB optimizer_002 STARTED
[03.04|15:54:25] Starting optimizer_001.prerun()
[03.04|15:54:25] Starting optimizer_002.prerun()
[03.04|15:54:25] optimizer_001.prerun() finished
[03.04|15:54:25] optimizer_002.prerun() finished
[03.04|15:54:25] JOB optimizer_002 RUNNING
[03.04|15:54:25] Executing optimizer_002.run
[03.04|15:54:25] JOB optimizer_001 RUNNING
[03.04|15:54:25] Waiting for job optimizer_001 to finish
[03.04|15:54:25] Executing optimizer_001.run
[03.04|15:55:40] training_set Optimizer: 002 Epoch: 0 Loss: 0.001776
[03.04|15:55:40] validation_set Optimizer: 002 Epoch: 0 Loss: 0.008428
[03.04|15:55:41] training_set Optimizer: 001 Epoch: 0 Loss: 0.002968
[03.04|15:55:41] validation_set Optimizer: 001 Epoch: 0 Loss: 0.026072
[03.04|15:55:50] training_set Optimizer: 002 Epoch: 10 Loss: 0.000577
[03.04|15:55:50] validation_set Optimizer: 002 Epoch: 10 Loss: 0.011576
[03.04|15:55:51] training_set Optimizer: 001 Epoch: 10 Loss: 0.000861
[03.04|15:55:51] validation_set Optimizer: 001 Epoch: 10 Loss: 0.015869
[03.04|15:55:59] training_set Optimizer: 002 Epoch: 20 Loss: 0.000737
[03.04|15:55:59] validation_set Optimizer: 002 Epoch: 20 Loss: 0.014445
[03.04|15:56:00] training_set Optimizer: 001 Epoch: 20 Loss: 0.000722
[03.04|15:56:00] validation_set Optimizer: 001 Epoch: 20 Loss: 0.018344
[03.04|15:56:08] training_set Optimizer: 002 Epoch: 30 Loss: 0.000627
[03.04|15:56:08] validation_set Optimizer: 002 Epoch: 30 Loss: 0.008928
[03.04|15:56:10] training_set Optimizer: 001 Epoch: 30 Loss: 0.000695
[03.04|15:56:10] validation_set Optimizer: 001 Epoch: 30 Loss: 0.013262
[03.04|15:56:17] training_set Optimizer: 002 Epoch: 40 Loss: 0.000614
[03.04|15:56:17] validation_set Optimizer: 002 Epoch: 40 Loss: 0.020043
[03.04|15:56:19] training_set Optimizer: 001 Epoch: 40 Loss: 0.000690
[03.04|15:56:19] validation_set Optimizer: 001 Epoch: 40 Loss: 0.011183
[03.04|15:56:27] training_set Optimizer: 002 Epoch: 50 Loss: 0.000520
[03.04|15:56:27] validation_set Optimizer: 002 Epoch: 50 Loss: 0.010220
[03.04|15:56:29] training_set Optimizer: 001 Epoch: 50 Loss: 0.000798
[03.04|15:56:29] validation_set Optimizer: 001 Epoch: 50 Loss: 0.031144
[03.04|15:56:36] training_set Optimizer: 002 Epoch: 60 Loss: 0.000436
[03.04|15:56:36] validation_set Optimizer: 002 Epoch: 60 Loss: 0.010977
[03.04|15:56:38] training_set Optimizer: 001 Epoch: 60 Loss: 0.000595
[03.04|15:56:38] validation_set Optimizer: 001 Epoch: 60 Loss: 0.010977
[03.04|15:56:45] training_set Optimizer: 002 Epoch: 70 Loss: 0.000521
[03.04|15:56:45] validation_set Optimizer: 002 Epoch: 70 Loss: 0.017383
[03.04|15:56:47] training_set Optimizer: 001 Epoch: 70 Loss: 0.000894
[03.04|15:56:47] validation_set Optimizer: 001 Epoch: 70 Loss: 0.010242
[03.04|15:56:54] training_set Optimizer: 002 Epoch: 80 Loss: 0.000546
[03.04|15:56:54] validation_set Optimizer: 002 Epoch: 80 Loss: 0.023210
[03.04|15:56:57] training_set Optimizer: 001 Epoch: 80 Loss: 0.000606
[03.04|15:56:57] validation_set Optimizer: 001 Epoch: 80 Loss: 0.010043
[03.04|15:57:04] training_set Optimizer: 002 Epoch: 90 Loss: 0.000424
[03.04|15:57:04] validation_set Optimizer: 002 Epoch: 90 Loss: 0.008804
[03.04|15:57:06] training_set Optimizer: 001 Epoch: 90 Loss: 0.000785
[03.04|15:57:06] validation_set Optimizer: 001 Epoch: 90 Loss: 0.010942
[03.04|15:57:13] training_set Optimizer: 002 Epoch: 100 Loss: 0.000551
[03.04|15:57:13] validation_set Optimizer: 002 Epoch: 100 Loss: 0.011911
[03.04|15:57:16] training_set Optimizer: 001 Epoch: 100 Loss: 0.000539
[03.04|15:57:16] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013295
[03.04|15:57:22] training_set Optimizer: 002 Epoch: 110 Loss: 0.000511
[03.04|15:57:22] validation_set Optimizer: 002 Epoch: 110 Loss: 0.018855
[03.04|15:57:26] training_set Optimizer: 001 Epoch: 110 Loss: 0.000559
[03.04|15:57:26] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011694
[03.04|15:57:33] Execution of optimizer_002.run finished with returncode 0
[03.04|15:57:33] JOB optimizer_002 FINISHED
[03.04|15:57:33] Starting optimizer_002.postrun()
[03.04|15:57:33] optimizer_002.postrun() finished
[03.04|15:57:33] JOB optimizer_002 SUCCESSFUL
[03.04|15:57:36] Execution of optimizer_001.run finished with returncode 0
[03.04|15:57:36] JOB optimizer_001 FINISHED
[03.04|15:57:36] Starting optimizer_001.postrun()
[03.04|15:57:36] optimizer_001.postrun() finished
[03.04|15:57:36] JOB optimizer_001 SUCCESSFUL
[03.04|15:57:36] PLAMS environment cleaned up successfully
[03.04|15:57:36] PLAMS run finished. Goodbye
[03.04|15:57:37] ParAMSResults
[03.04|15:57:37] Newly created parameter file/dir: step5_attempt7_training/results/optimization/optimizer_001/m3gnet
[03.04|15:57:37] Newly created parameter file/dir: step5_attempt7_training/results/optimization/optimizer_002/m3gnet
[03.04|15:57:37] Done!
[03.04|15:57:37] Deleting step5_attempt6_training
[03.04|15:57:37] ##########################
[03.04|15:57:37] ### Step 5 / Attempt 8 ###
[03.04|15:57:37] ##########################
[03.04|15:57:37] MD Steps: 2280 (cumulative: 3000)
[03.04|15:57:37] Current engine settings:
[03.04|15:57:37]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt7_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt7_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|15:57:37] Running step5_attempt8_simulation...
[03.04|15:58:39] Job step5_attempt8_simulation finished
[03.04|15:58:39] Deleting files that are no longer needed...
[03.04|15:58:39] Energy uncertainty for final frame of step5_attempt8_simulation: 0.2758 eV
[03.04|15:58:39] 0.0230 eV/atom
[03.04|15:58:39] Forces uncertainty for final frame of step5_attempt8_simulation: 1.0083 eV/angstrom
[03.04|15:58:40] Launching reference calculation
[03.04|15:58:54] Reference calculation finished!
[03.04|15:58:54] Adding results from step5_attempt8_reference_calc1 to training set
[03.04|15:58:54] Current # training set entries: 72
[03.04|15:58:54] Current # validation set entries: 21
[03.04|15:58:54] Storing data in step5_attempt8_reference_data
[03.04|15:58:54] Deleting step5_attempt7_reference_data
[03.04|15:58:54] Deleting step5_attempt8_reference_calc1
[03.04|15:58:54]
[03.04|15:58:54] Current (cumulative) timings:
[03.04|15:58:54] Time (s) Fraction
[03.04|15:58:54] Ref. calcs 776.21 0.146
[03.04|15:58:54] ML training 3407.40 0.640
[03.04|15:58:54] Simulations 1140.94 0.214
[03.04|15:58:54]
[03.04|15:58:54]
[03.04|15:58:54]
[03.04|15:58:54] --- Begin summary ---
[03.04|15:58:54] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|15:58:54] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|15:58:54] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|15:58:54] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|15:58:54] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|15:58:54] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|15:58:54] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|15:58:54] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|15:58:54] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|15:58:54] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|15:58:54] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|15:58:54] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|15:58:54] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|15:58:54] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|15:58:54] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|15:58:54] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|15:58:54] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|15:58:54] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|15:58:54] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|15:58:54] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|15:58:54] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|15:58:54] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|15:58:54] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|15:58:54] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|15:58:54] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|15:58:54] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|15:58:54] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|15:58:54] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|15:58:54] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|15:58:54] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|15:58:54] --- End summary ---
[03.04|15:58:54]
[03.04|15:58:54] Running more reference calculations....
[03.04|15:58:55] Running reference calculations on frames [173, 215] from step5_attempt8_simulation/ams.rkf
[03.04|15:58:55] Calculating 2 frames in total
[03.04|15:58:55] Running step5_attempt8_reference_calc2
[03.04|15:59:09] Running step5_attempt8_reference_calc3
[03.04|15:59:23] Reference calculations finished!
[03.04|15:59:23] Adding results from step5_attempt8_reference_calc2 to training set
[03.04|15:59:23] Adding results from step5_attempt8_reference_calc3 to training set
[03.04|15:59:23] Current # training set entries: 74
[03.04|15:59:23] Current # validation set entries: 21
[03.04|15:59:23] Storing data in step5_attempt8_reference_data
[03.04|15:59:23] Deleting step5_attempt8_reference_calc2
[03.04|15:59:23] Deleting step5_attempt8_reference_calc3
[03.04|15:59:23] Launching reparametrization job: step5_attempt8_training
[03.04|15:59:28] JOB optimizer_001 STARTED
[03.04|15:59:28] JOB optimizer_002 STARTED
[03.04|15:59:28] Starting optimizer_001.prerun()
[03.04|15:59:28] Starting optimizer_002.prerun()
[03.04|15:59:28] optimizer_002.prerun() finished
[03.04|15:59:28] optimizer_001.prerun() finished
[03.04|15:59:28] JOB optimizer_001 RUNNING
[03.04|15:59:28] Executing optimizer_001.run
[03.04|15:59:28] JOB optimizer_002 RUNNING
[03.04|15:59:28] Executing optimizer_002.run
[03.04|15:59:28] Waiting for job optimizer_001 to finish
[03.04|16:00:42] training_set Optimizer: 001 Epoch: 0 Loss: 0.003330
[03.04|16:00:42] validation_set Optimizer: 001 Epoch: 0 Loss: 0.017190
[03.04|16:00:44] training_set Optimizer: 002 Epoch: 0 Loss: 0.002099
[03.04|16:00:44] validation_set Optimizer: 002 Epoch: 0 Loss: 0.012318
[03.04|16:00:52] training_set Optimizer: 001 Epoch: 10 Loss: 0.000849
[03.04|16:00:52] validation_set Optimizer: 001 Epoch: 10 Loss: 0.011342
[03.04|16:00:53] training_set Optimizer: 002 Epoch: 10 Loss: 0.000879
[03.04|16:00:53] validation_set Optimizer: 002 Epoch: 10 Loss: 0.017506
[03.04|16:01:01] training_set Optimizer: 001 Epoch: 20 Loss: 0.000824
[03.04|16:01:01] validation_set Optimizer: 001 Epoch: 20 Loss: 0.012422
[03.04|16:01:02] training_set Optimizer: 002 Epoch: 20 Loss: 0.000631
[03.04|16:01:02] validation_set Optimizer: 002 Epoch: 20 Loss: 0.011370
[03.04|16:01:11] training_set Optimizer: 001 Epoch: 30 Loss: 0.000730
[03.04|16:01:11] validation_set Optimizer: 001 Epoch: 30 Loss: 0.021629
[03.04|16:01:12] training_set Optimizer: 002 Epoch: 30 Loss: 0.000491
[03.04|16:01:12] validation_set Optimizer: 002 Epoch: 30 Loss: 0.013732
[03.04|16:01:20] training_set Optimizer: 001 Epoch: 40 Loss: 0.000639
[03.04|16:01:20] validation_set Optimizer: 001 Epoch: 40 Loss: 0.012027
[03.04|16:01:21] training_set Optimizer: 002 Epoch: 40 Loss: 0.000651
[03.04|16:01:21] validation_set Optimizer: 002 Epoch: 40 Loss: 0.008190
[03.04|16:01:29] training_set Optimizer: 001 Epoch: 50 Loss: 0.000866
[03.04|16:01:29] validation_set Optimizer: 001 Epoch: 50 Loss: 0.012691
[03.04|16:01:31] training_set Optimizer: 002 Epoch: 50 Loss: 0.000408
[03.04|16:01:31] validation_set Optimizer: 002 Epoch: 50 Loss: 0.017536
[03.04|16:01:39] training_set Optimizer: 001 Epoch: 60 Loss: 0.000908
[03.04|16:01:39] validation_set Optimizer: 001 Epoch: 60 Loss: 0.022894
[03.04|16:01:40] training_set Optimizer: 002 Epoch: 60 Loss: 0.000403
[03.04|16:01:40] validation_set Optimizer: 002 Epoch: 60 Loss: 0.015932
[03.04|16:01:48] training_set Optimizer: 001 Epoch: 70 Loss: 0.000508
[03.04|16:01:48] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011639
[03.04|16:01:50] training_set Optimizer: 002 Epoch: 70 Loss: 0.000443
[03.04|16:01:50] validation_set Optimizer: 002 Epoch: 70 Loss: 0.010487
[03.04|16:01:58] training_set Optimizer: 001 Epoch: 80 Loss: 0.000650
[03.04|16:01:58] validation_set Optimizer: 001 Epoch: 80 Loss: 0.011145
[03.04|16:01:59] training_set Optimizer: 002 Epoch: 80 Loss: 0.000376
[03.04|16:01:59] validation_set Optimizer: 002 Epoch: 80 Loss: 0.007504
[03.04|16:02:07] training_set Optimizer: 001 Epoch: 90 Loss: 0.000540
[03.04|16:02:07] validation_set Optimizer: 001 Epoch: 90 Loss: 0.010726
[03.04|16:02:08] training_set Optimizer: 002 Epoch: 90 Loss: 0.000489
[03.04|16:02:08] validation_set Optimizer: 002 Epoch: 90 Loss: 0.007658
[03.04|16:02:17] training_set Optimizer: 001 Epoch: 100 Loss: 0.000556
[03.04|16:02:17] validation_set Optimizer: 001 Epoch: 100 Loss: 0.011358
[03.04|16:02:18] training_set Optimizer: 002 Epoch: 100 Loss: 0.000489
[03.04|16:02:18] validation_set Optimizer: 002 Epoch: 100 Loss: 0.009589
[03.04|16:02:26] training_set Optimizer: 001 Epoch: 110 Loss: 0.000560
[03.04|16:02:26] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011732
[03.04|16:02:28] training_set Optimizer: 002 Epoch: 110 Loss: 0.000451
[03.04|16:02:28] validation_set Optimizer: 002 Epoch: 110 Loss: 0.020907
[03.04|16:02:36] Execution of optimizer_001.run finished with returncode 0
[03.04|16:02:37] JOB optimizer_001 FINISHED
[03.04|16:02:37] Starting optimizer_001.postrun()
[03.04|16:02:37] optimizer_001.postrun() finished
[03.04|16:02:37] JOB optimizer_001 SUCCESSFUL
[03.04|16:02:37] Waiting for job optimizer_002 to finish
[03.04|16:02:38] Execution of optimizer_002.run finished with returncode 0
[03.04|16:02:38] JOB optimizer_002 FINISHED
[03.04|16:02:38] Starting optimizer_002.postrun()
[03.04|16:02:38] optimizer_002.postrun() finished
[03.04|16:02:38] JOB optimizer_002 SUCCESSFUL
[03.04|16:02:38] PLAMS environment cleaned up successfully
[03.04|16:02:38] PLAMS run finished. Goodbye
[03.04|16:02:39] ParAMSResults
[03.04|16:02:39] Newly created parameter file/dir: step5_attempt8_training/results/optimization/optimizer_001/m3gnet
[03.04|16:02:39] Newly created parameter file/dir: step5_attempt8_training/results/optimization/optimizer_002/m3gnet
[03.04|16:02:39] Done!
[03.04|16:02:39] Deleting step5_attempt7_training
[03.04|16:02:39] ##########################
[03.04|16:02:39] ### Step 5 / Attempt 9 ###
[03.04|16:02:39] ##########################
[03.04|16:02:39] MD Steps: 2280 (cumulative: 3000)
[03.04|16:02:39] Current engine settings:
[03.04|16:02:39]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt8_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt8_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:02:39] Running step5_attempt9_simulation...
[03.04|16:03:44] Job step5_attempt9_simulation finished
[03.04|16:03:44] Deleting files that are no longer needed...
[03.04|16:03:44] Energy uncertainty for final frame of step5_attempt9_simulation: 0.0200 eV
[03.04|16:03:44] 0.0017 eV/atom
[03.04|16:03:44] Forces uncertainty for final frame of step5_attempt9_simulation: 1.0508 eV/angstrom
[03.04|16:03:45] Launching reference calculation
[03.04|16:03:59] Reference calculation finished!
[03.04|16:03:59] Adding results from step5_attempt9_reference_calc1 to training set
[03.04|16:03:59] Current # training set entries: 75
[03.04|16:03:59] Current # validation set entries: 21
[03.04|16:03:59] Storing data in step5_attempt9_reference_data
[03.04|16:03:59] Deleting step5_attempt8_reference_data
[03.04|16:03:59] Deleting step5_attempt9_reference_calc1
[03.04|16:03:59]
[03.04|16:03:59] Current (cumulative) timings:
[03.04|16:03:59] Time (s) Fraction
[03.04|16:03:59] Ref. calcs 818.65 0.145
[03.04|16:03:59] ML training 3602.99 0.640
[03.04|16:03:59] Simulations 1206.15 0.214
[03.04|16:03:59]
[03.04|16:03:59]
[03.04|16:03:59]
[03.04|16:03:59] --- Begin summary ---
[03.04|16:03:59] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:03:59] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:03:59] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:03:59] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:03:59] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:03:59] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:03:59] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:03:59] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:03:59] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:03:59] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:03:59] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:03:59] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:03:59] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:03:59] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:03:59] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:03:59] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:03:59] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:03:59] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:03:59] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:03:59] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:03:59] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:03:59] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:03:59] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:03:59] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:03:59] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:03:59] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:03:59] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:03:59] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:03:59] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:03:59] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:03:59] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:03:59] --- End summary ---
[03.04|16:03:59]
[03.04|16:03:59] Running more reference calculations....
[03.04|16:04:00] Running reference calculations on frames [137, 184] from step5_attempt9_simulation/ams.rkf
[03.04|16:04:00] Calculating 2 frames in total
[03.04|16:04:00] Running step5_attempt9_reference_calc2
[03.04|16:04:14] Running step5_attempt9_reference_calc3
[03.04|16:04:28] Reference calculations finished!
[03.04|16:04:28] Adding results from step5_attempt9_reference_calc2 to training set
[03.04|16:04:28] Adding results from step5_attempt9_reference_calc3 to training set
[03.04|16:04:28] Current # training set entries: 77
[03.04|16:04:28] Current # validation set entries: 21
[03.04|16:04:28] Storing data in step5_attempt9_reference_data
[03.04|16:04:29] Deleting step5_attempt9_reference_calc2
[03.04|16:04:29] Deleting step5_attempt9_reference_calc3
[03.04|16:04:29] Launching reparametrization job: step5_attempt9_training
[03.04|16:04:33] JOB optimizer_001 STARTED
[03.04|16:04:33] Starting optimizer_001.prerun()
[03.04|16:04:33] optimizer_001.prerun() finished
[03.04|16:04:33] JOB optimizer_002 STARTED
[03.04|16:04:33] Starting optimizer_002.prerun()
[03.04|16:04:33] optimizer_002.prerun() finished
[03.04|16:04:34] JOB optimizer_002 RUNNING
[03.04|16:04:34] JOB optimizer_001 RUNNING
[03.04|16:04:34] Executing optimizer_002.run
[03.04|16:04:34] Executing optimizer_001.run
[03.04|16:04:34] Waiting for job optimizer_001 to finish
[03.04|16:05:48] training_set Optimizer: 002 Epoch: 0 Loss: 0.002330
[03.04|16:05:48] validation_set Optimizer: 002 Epoch: 0 Loss: 0.014621
[03.04|16:05:49] training_set Optimizer: 001 Epoch: 0 Loss: 0.001604
[03.04|16:05:49] validation_set Optimizer: 001 Epoch: 0 Loss: 0.012425
[03.04|16:05:58] training_set Optimizer: 002 Epoch: 10 Loss: 0.000442
[03.04|16:05:58] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009238
[03.04|16:05:59] training_set Optimizer: 001 Epoch: 10 Loss: 0.000807
[03.04|16:05:59] validation_set Optimizer: 001 Epoch: 10 Loss: 0.019874
[03.04|16:06:08] training_set Optimizer: 002 Epoch: 20 Loss: 0.000591
[03.04|16:06:08] validation_set Optimizer: 002 Epoch: 20 Loss: 0.011567
[03.04|16:06:09] training_set Optimizer: 001 Epoch: 20 Loss: 0.001000
[03.04|16:06:09] validation_set Optimizer: 001 Epoch: 20 Loss: 0.014480
[03.04|16:06:18] training_set Optimizer: 002 Epoch: 30 Loss: 0.000434
[03.04|16:06:18] validation_set Optimizer: 002 Epoch: 30 Loss: 0.007864
[03.04|16:06:19] training_set Optimizer: 001 Epoch: 30 Loss: 0.001039
[03.04|16:06:19] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014774
[03.04|16:06:28] training_set Optimizer: 002 Epoch: 40 Loss: 0.000673
[03.04|16:06:28] validation_set Optimizer: 002 Epoch: 40 Loss: 0.010411
[03.04|16:06:29] training_set Optimizer: 001 Epoch: 40 Loss: 0.000508
[03.04|16:06:29] validation_set Optimizer: 001 Epoch: 40 Loss: 0.010775
[03.04|16:06:38] training_set Optimizer: 002 Epoch: 50 Loss: 0.000570
[03.04|16:06:38] validation_set Optimizer: 002 Epoch: 50 Loss: 0.007333
[03.04|16:06:39] training_set Optimizer: 001 Epoch: 50 Loss: 0.000507
[03.04|16:06:39] validation_set Optimizer: 001 Epoch: 50 Loss: 0.015652
[03.04|16:06:48] training_set Optimizer: 002 Epoch: 60 Loss: 0.000407
[03.04|16:06:48] validation_set Optimizer: 002 Epoch: 60 Loss: 0.008502
[03.04|16:06:49] training_set Optimizer: 001 Epoch: 60 Loss: 0.000597
[03.04|16:06:49] validation_set Optimizer: 001 Epoch: 60 Loss: 0.011848
[03.04|16:06:58] training_set Optimizer: 002 Epoch: 70 Loss: 0.000404
[03.04|16:06:58] validation_set Optimizer: 002 Epoch: 70 Loss: 0.007532
[03.04|16:06:59] training_set Optimizer: 001 Epoch: 70 Loss: 0.000550
[03.04|16:06:59] validation_set Optimizer: 001 Epoch: 70 Loss: 0.015482
[03.04|16:07:08] training_set Optimizer: 002 Epoch: 80 Loss: 0.000408
[03.04|16:07:08] validation_set Optimizer: 002 Epoch: 80 Loss: 0.007249
[03.04|16:07:09] training_set Optimizer: 001 Epoch: 80 Loss: 0.000580
[03.04|16:07:09] validation_set Optimizer: 001 Epoch: 80 Loss: 0.016414
[03.04|16:07:18] training_set Optimizer: 002 Epoch: 90 Loss: 0.000330
[03.04|16:07:18] validation_set Optimizer: 002 Epoch: 90 Loss: 0.010955
[03.04|16:07:19] training_set Optimizer: 001 Epoch: 90 Loss: 0.000464
[03.04|16:07:19] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013118
[03.04|16:07:28] training_set Optimizer: 002 Epoch: 100 Loss: 0.000373
[03.04|16:07:28] validation_set Optimizer: 002 Epoch: 100 Loss: 0.007246
[03.04|16:07:29] training_set Optimizer: 001 Epoch: 100 Loss: 0.000422
[03.04|16:07:29] validation_set Optimizer: 001 Epoch: 100 Loss: 0.011905
[03.04|16:07:38] training_set Optimizer: 002 Epoch: 110 Loss: 0.000330
[03.04|16:07:38] validation_set Optimizer: 002 Epoch: 110 Loss: 0.008199
[03.04|16:07:39] training_set Optimizer: 001 Epoch: 110 Loss: 0.000617
[03.04|16:07:39] validation_set Optimizer: 001 Epoch: 110 Loss: 0.027294
[03.04|16:07:49] Execution of optimizer_002.run finished with returncode 0
[03.04|16:07:49] JOB optimizer_002 FINISHED
[03.04|16:07:49] Starting optimizer_002.postrun()
[03.04|16:07:49] optimizer_002.postrun() finished
[03.04|16:07:50] JOB optimizer_002 SUCCESSFUL
[03.04|16:07:50] Execution of optimizer_001.run finished with returncode 0
[03.04|16:07:50] JOB optimizer_001 FINISHED
[03.04|16:07:50] Starting optimizer_001.postrun()
[03.04|16:07:50] optimizer_001.postrun() finished
[03.04|16:07:51] JOB optimizer_001 SUCCESSFUL
[03.04|16:07:51] PLAMS environment cleaned up successfully
[03.04|16:07:51] PLAMS run finished. Goodbye
[03.04|16:07:51] ParAMSResults
[03.04|16:07:51] Newly created parameter file/dir: step5_attempt9_training/results/optimization/optimizer_001/m3gnet
[03.04|16:07:51] Newly created parameter file/dir: step5_attempt9_training/results/optimization/optimizer_002/m3gnet
[03.04|16:07:51] Done!
[03.04|16:07:51] Deleting step5_attempt8_training
[03.04|16:07:51] ###########################
[03.04|16:07:51] ### Step 5 / Attempt 10 ###
[03.04|16:07:51] ###########################
[03.04|16:07:51] MD Steps: 2280 (cumulative: 3000)
[03.04|16:07:51] Current engine settings:
[03.04|16:07:51]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt9_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt9_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:07:51] Running step5_attempt10_simulation...
[03.04|16:08:56] Job step5_attempt10_simulation finished
[03.04|16:08:56] Deleting files that are no longer needed...
[03.04|16:08:56] Energy uncertainty for final frame of step5_attempt10_simulation: 0.0620 eV
[03.04|16:08:56] 0.0052 eV/atom
[03.04|16:08:56] Forces uncertainty for final frame of step5_attempt10_simulation: 1.0253 eV/angstrom
[03.04|16:08:57] Launching reference calculation
[03.04|16:09:11] Reference calculation finished!
[03.04|16:09:11] Adding results from step5_attempt10_reference_calc1 to training set
[03.04|16:09:11] Current # training set entries: 78
[03.04|16:09:11] Current # validation set entries: 21
[03.04|16:09:11] Storing data in step5_attempt10_reference_data
[03.04|16:09:11] Deleting step5_attempt9_reference_data
[03.04|16:09:11] Deleting step5_attempt10_reference_calc1
[03.04|16:09:11]
[03.04|16:09:11] Current (cumulative) timings:
[03.04|16:09:11] Time (s) Fraction
[03.04|16:09:11] Ref. calcs 861.45 0.145
[03.04|16:09:11] ML training 3805.64 0.641
[03.04|16:09:11] Simulations 1270.96 0.214
[03.04|16:09:11]
[03.04|16:09:11]
[03.04|16:09:11]
[03.04|16:09:11] --- Begin summary ---
[03.04|16:09:11] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:09:11] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:09:11] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:09:11] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:09:11] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:09:11] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:09:11] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:09:11] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:09:11] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:09:11] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:09:11] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:09:11] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:09:11] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:09:11] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:09:11] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:09:11] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:09:11] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:09:11] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:09:11] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:09:11] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:09:11] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:09:11] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:09:11] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:09:11] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:09:11] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:09:11] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:09:11] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:09:11] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:09:11] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:09:11] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:09:11] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:09:11] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:09:11] --- End summary ---
[03.04|16:09:11]
[03.04|16:09:11] Running more reference calculations....
[03.04|16:09:12] Running reference calculations on frames [136, 183] from step5_attempt10_simulation/ams.rkf
[03.04|16:09:12] Calculating 2 frames in total
[03.04|16:09:12] Running step5_attempt10_reference_calc2
[03.04|16:09:25] Running step5_attempt10_reference_calc3
[03.04|16:09:39] Reference calculations finished!
[03.04|16:09:39] Adding results from step5_attempt10_reference_calc2 to training set
[03.04|16:09:39] Adding results from step5_attempt10_reference_calc3 to training set
[03.04|16:09:39] Current # training set entries: 80
[03.04|16:09:39] Current # validation set entries: 21
[03.04|16:09:39] Storing data in step5_attempt10_reference_data
[03.04|16:09:40] Deleting step5_attempt10_reference_calc2
[03.04|16:09:40] Deleting step5_attempt10_reference_calc3
[03.04|16:09:40] Launching reparametrization job: step5_attempt10_training
[03.04|16:09:44] JOB optimizer_001 STARTED
[03.04|16:09:44] JOB optimizer_002 STARTED
[03.04|16:09:44] Starting optimizer_001.prerun()
[03.04|16:09:44] optimizer_001.prerun() finished
[03.04|16:09:44] Starting optimizer_002.prerun()
[03.04|16:09:44] optimizer_002.prerun() finished
[03.04|16:09:45] JOB optimizer_002 RUNNING
[03.04|16:09:45] JOB optimizer_001 RUNNING
[03.04|16:09:45] Executing optimizer_002.run
[03.04|16:09:45] Executing optimizer_001.run
[03.04|16:09:45] Waiting for job optimizer_001 to finish
[03.04|16:10:38] training_set Optimizer: 001 Epoch: 0 Loss: 0.002160
[03.04|16:10:38] validation_set Optimizer: 001 Epoch: 0 Loss: 0.018029
[03.04|16:10:39] training_set Optimizer: 002 Epoch: 0 Loss: 0.001749
[03.04|16:10:39] validation_set Optimizer: 002 Epoch: 0 Loss: 0.015106
[03.04|16:10:48] training_set Optimizer: 001 Epoch: 10 Loss: 0.000504
[03.04|16:10:48] validation_set Optimizer: 001 Epoch: 10 Loss: 0.011831
[03.04|16:10:49] training_set Optimizer: 002 Epoch: 10 Loss: 0.000533
[03.04|16:10:49] validation_set Optimizer: 002 Epoch: 10 Loss: 0.006573
[03.04|16:10:58] training_set Optimizer: 001 Epoch: 20 Loss: 0.000531
[03.04|16:10:58] validation_set Optimizer: 001 Epoch: 20 Loss: 0.013643
[03.04|16:10:59] training_set Optimizer: 002 Epoch: 20 Loss: 0.000421
[03.04|16:10:59] validation_set Optimizer: 002 Epoch: 20 Loss: 0.015369
[03.04|16:11:08] training_set Optimizer: 001 Epoch: 30 Loss: 0.000530
[03.04|16:11:08] validation_set Optimizer: 001 Epoch: 30 Loss: 0.011876
[03.04|16:11:09] training_set Optimizer: 002 Epoch: 30 Loss: 0.000395
[03.04|16:11:09] validation_set Optimizer: 002 Epoch: 30 Loss: 0.011819
[03.04|16:11:17] training_set Optimizer: 001 Epoch: 40 Loss: 0.000510
[03.04|16:11:17] validation_set Optimizer: 001 Epoch: 40 Loss: 0.013746
[03.04|16:11:18] training_set Optimizer: 002 Epoch: 40 Loss: 0.000345
[03.04|16:11:18] validation_set Optimizer: 002 Epoch: 40 Loss: 0.006787
[03.04|16:11:27] training_set Optimizer: 001 Epoch: 50 Loss: 0.000481
[03.04|16:11:27] validation_set Optimizer: 001 Epoch: 50 Loss: 0.010561
[03.04|16:11:28] training_set Optimizer: 002 Epoch: 50 Loss: 0.000464
[03.04|16:11:28] validation_set Optimizer: 002 Epoch: 50 Loss: 0.010930
[03.04|16:11:37] training_set Optimizer: 001 Epoch: 60 Loss: 0.000501
[03.04|16:11:37] validation_set Optimizer: 001 Epoch: 60 Loss: 0.010840
[03.04|16:11:38] training_set Optimizer: 002 Epoch: 60 Loss: 0.000343
[03.04|16:11:38] validation_set Optimizer: 002 Epoch: 60 Loss: 0.006895
[03.04|16:11:47] training_set Optimizer: 001 Epoch: 70 Loss: 0.000471
[03.04|16:11:47] validation_set Optimizer: 001 Epoch: 70 Loss: 0.010612
[03.04|16:11:48] training_set Optimizer: 002 Epoch: 70 Loss: 0.000409
[03.04|16:11:48] validation_set Optimizer: 002 Epoch: 70 Loss: 0.010244
[03.04|16:11:57] training_set Optimizer: 001 Epoch: 80 Loss: 0.000667
[03.04|16:11:57] validation_set Optimizer: 001 Epoch: 80 Loss: 0.024431
[03.04|16:11:58] training_set Optimizer: 002 Epoch: 80 Loss: 0.000394
[03.04|16:11:58] validation_set Optimizer: 002 Epoch: 80 Loss: 0.006979
[03.04|16:12:07] training_set Optimizer: 001 Epoch: 90 Loss: 0.000393
[03.04|16:12:07] validation_set Optimizer: 001 Epoch: 90 Loss: 0.009972
[03.04|16:12:08] training_set Optimizer: 002 Epoch: 90 Loss: 0.000320
[03.04|16:12:08] validation_set Optimizer: 002 Epoch: 90 Loss: 0.012130
[03.04|16:12:17] training_set Optimizer: 001 Epoch: 100 Loss: 0.000385
[03.04|16:12:17] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013016
[03.04|16:12:18] training_set Optimizer: 002 Epoch: 100 Loss: 0.000304
[03.04|16:12:18] validation_set Optimizer: 002 Epoch: 100 Loss: 0.010192
[03.04|16:12:27] training_set Optimizer: 001 Epoch: 110 Loss: 0.000467
[03.04|16:12:27] validation_set Optimizer: 001 Epoch: 110 Loss: 0.012895
[03.04|16:12:28] training_set Optimizer: 002 Epoch: 110 Loss: 0.000502
[03.04|16:12:28] validation_set Optimizer: 002 Epoch: 110 Loss: 0.010384
[03.04|16:12:37] Execution of optimizer_001.run finished with returncode 0
[03.04|16:12:37] JOB optimizer_001 FINISHED
[03.04|16:12:37] Starting optimizer_001.postrun()
[03.04|16:12:37] optimizer_001.postrun() finished
[03.04|16:12:38] JOB optimizer_001 SUCCESSFUL
[03.04|16:12:38] Waiting for job optimizer_002 to finish
[03.04|16:12:38] Execution of optimizer_002.run finished with returncode 0
[03.04|16:12:38] JOB optimizer_002 FINISHED
[03.04|16:12:38] Starting optimizer_002.postrun()
[03.04|16:12:38] optimizer_002.postrun() finished
[03.04|16:12:39] JOB optimizer_002 SUCCESSFUL
[03.04|16:12:39] PLAMS environment cleaned up successfully
[03.04|16:12:39] PLAMS run finished. Goodbye
[03.04|16:12:39] ParAMSResults
[03.04|16:12:39] Newly created parameter file/dir: step5_attempt10_training/results/optimization/optimizer_001/m3gnet
[03.04|16:12:39] Newly created parameter file/dir: step5_attempt10_training/results/optimization/optimizer_002/m3gnet
[03.04|16:12:39] Done!
[03.04|16:12:39] Deleting step5_attempt9_training
[03.04|16:12:39] ###########################
[03.04|16:12:39] ### Step 5 / Attempt 11 ###
[03.04|16:12:39] ###########################
[03.04|16:12:39] MD Steps: 2280 (cumulative: 3000)
[03.04|16:12:39] Current engine settings:
[03.04|16:12:39]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt10_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt10_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:12:39] Running step5_attempt11_simulation...
[03.04|16:13:45] Job step5_attempt11_simulation finished
[03.04|16:13:45] Deleting files that are no longer needed...
[03.04|16:13:45] Energy uncertainty for final frame of step5_attempt11_simulation: 0.0055 eV
[03.04|16:13:45] 0.0005 eV/atom
[03.04|16:13:45] Forces uncertainty for final frame of step5_attempt11_simulation: 1.0613 eV/angstrom
[03.04|16:13:45] Launching reference calculation
[03.04|16:14:00] Reference calculation finished!
[03.04|16:14:00] Adding results from step5_attempt11_reference_calc1 to training set
[03.04|16:14:00] Current # training set entries: 81
[03.04|16:14:00] Current # validation set entries: 21
[03.04|16:14:00] Storing data in step5_attempt11_reference_data
[03.04|16:14:00] Deleting step5_attempt10_reference_data
[03.04|16:14:00] Deleting step5_attempt11_reference_calc1
[03.04|16:14:00]
[03.04|16:14:00] Current (cumulative) timings:
[03.04|16:14:00] Time (s) Fraction
[03.04|16:14:00] Ref. calcs 903.41 0.145
[03.04|16:14:00] ML training 3985.23 0.640
[03.04|16:14:00] Simulations 1336.48 0.215
[03.04|16:14:00]
[03.04|16:14:00]
[03.04|16:14:00]
[03.04|16:14:00] --- Begin summary ---
[03.04|16:14:00] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:14:00] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:14:00] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:14:00] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:14:00] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:14:00] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:14:00] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:14:00] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:14:00] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:14:00] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:14:00] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:14:00] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:14:00] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:14:00] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:14:00] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:14:00] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:14:00] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:14:00] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:14:00] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:14:00] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:14:00] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:14:00] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:14:00] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:14:00] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:14:00] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:14:00] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:14:00] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:14:00] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:14:00] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:14:00] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:14:00] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:14:00] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:14:00] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:14:00] --- End summary ---
[03.04|16:14:00]
[03.04|16:14:00] Running more reference calculations....
[03.04|16:14:01] Running reference calculations on frames [137, 184] from step5_attempt11_simulation/ams.rkf
[03.04|16:14:01] Calculating 2 frames in total
[03.04|16:14:01] Running step5_attempt11_reference_calc2
[03.04|16:14:15] Running step5_attempt11_reference_calc3
[03.04|16:14:28] Reference calculations finished!
[03.04|16:14:29] Adding results from step5_attempt11_reference_calc2 to training set
[03.04|16:14:29] Adding results from step5_attempt11_reference_calc3 to training set
[03.04|16:14:29] Current # training set entries: 83
[03.04|16:14:29] Current # validation set entries: 21
[03.04|16:14:29] Storing data in step5_attempt11_reference_data
[03.04|16:14:29] Deleting step5_attempt11_reference_calc2
[03.04|16:14:29] Deleting step5_attempt11_reference_calc3
[03.04|16:14:29] Launching reparametrization job: step5_attempt11_training
[03.04|16:14:34] JOB optimizer_001 STARTED
[03.04|16:14:34] Starting optimizer_001.prerun()
[03.04|16:14:34] JOB optimizer_002 STARTED
[03.04|16:14:34] optimizer_001.prerun() finished
[03.04|16:14:34] Starting optimizer_002.prerun()
[03.04|16:14:34] optimizer_002.prerun() finished
[03.04|16:14:34] JOB optimizer_001 RUNNING
[03.04|16:14:34] Executing optimizer_001.run
[03.04|16:14:34] Waiting for job optimizer_001 to finish
[03.04|16:14:34] JOB optimizer_002 RUNNING
[03.04|16:14:34] Executing optimizer_002.run
[03.04|16:15:50] training_set Optimizer: 001 Epoch: 0 Loss: 0.001791
[03.04|16:15:50] validation_set Optimizer: 001 Epoch: 0 Loss: 0.019218
[03.04|16:15:50] training_set Optimizer: 002 Epoch: 0 Loss: 0.001589
[03.04|16:15:50] validation_set Optimizer: 002 Epoch: 0 Loss: 0.017031
[03.04|16:16:00] training_set Optimizer: 001 Epoch: 10 Loss: 0.000449
[03.04|16:16:00] validation_set Optimizer: 001 Epoch: 10 Loss: 0.010637
[03.04|16:16:00] training_set Optimizer: 002 Epoch: 10 Loss: 0.000332
[03.04|16:16:00] validation_set Optimizer: 002 Epoch: 10 Loss: 0.008022
[03.04|16:16:11] training_set Optimizer: 002 Epoch: 20 Loss: 0.000491
[03.04|16:16:11] validation_set Optimizer: 002 Epoch: 20 Loss: 0.013808
[03.04|16:16:11] training_set Optimizer: 001 Epoch: 20 Loss: 0.000629
[03.04|16:16:11] validation_set Optimizer: 001 Epoch: 20 Loss: 0.016544
[03.04|16:16:22] training_set Optimizer: 001 Epoch: 30 Loss: 0.000684
[03.04|16:16:22] validation_set Optimizer: 001 Epoch: 30 Loss: 0.020154
[03.04|16:16:22] training_set Optimizer: 002 Epoch: 30 Loss: 0.000463
[03.04|16:16:22] validation_set Optimizer: 002 Epoch: 30 Loss: 0.011225
[03.04|16:16:32] training_set Optimizer: 001 Epoch: 40 Loss: 0.000484
[03.04|16:16:32] validation_set Optimizer: 001 Epoch: 40 Loss: 0.010692
[03.04|16:16:32] training_set Optimizer: 002 Epoch: 40 Loss: 0.000289
[03.04|16:16:32] validation_set Optimizer: 002 Epoch: 40 Loss: 0.006122
[03.04|16:16:43] training_set Optimizer: 001 Epoch: 50 Loss: 0.000445
[03.04|16:16:43] validation_set Optimizer: 001 Epoch: 50 Loss: 0.011889
[03.04|16:16:43] training_set Optimizer: 002 Epoch: 50 Loss: 0.000321
[03.04|16:16:43] validation_set Optimizer: 002 Epoch: 50 Loss: 0.009211
[03.04|16:16:53] training_set Optimizer: 001 Epoch: 60 Loss: 0.000955
[03.04|16:16:53] validation_set Optimizer: 001 Epoch: 60 Loss: 0.016940
[03.04|16:16:53] training_set Optimizer: 002 Epoch: 60 Loss: 0.000325
[03.04|16:16:53] validation_set Optimizer: 002 Epoch: 60 Loss: 0.008833
[03.04|16:17:03] training_set Optimizer: 001 Epoch: 70 Loss: 0.000426
[03.04|16:17:03] validation_set Optimizer: 001 Epoch: 70 Loss: 0.010126
[03.04|16:17:04] training_set Optimizer: 002 Epoch: 70 Loss: 0.000273
[03.04|16:17:04] validation_set Optimizer: 002 Epoch: 70 Loss: 0.011208
[03.04|16:17:14] training_set Optimizer: 001 Epoch: 80 Loss: 0.000385
[03.04|16:17:14] validation_set Optimizer: 001 Epoch: 80 Loss: 0.008946
[03.04|16:17:14] training_set Optimizer: 002 Epoch: 80 Loss: 0.000563
[03.04|16:17:14] validation_set Optimizer: 002 Epoch: 80 Loss: 0.007590
[03.04|16:17:24] training_set Optimizer: 001 Epoch: 90 Loss: 0.001229
[03.04|16:17:24] validation_set Optimizer: 001 Epoch: 90 Loss: 0.011910
[03.04|16:17:25] training_set Optimizer: 002 Epoch: 90 Loss: 0.000462
[03.04|16:17:25] validation_set Optimizer: 002 Epoch: 90 Loss: 0.007383
[03.04|16:17:35] training_set Optimizer: 001 Epoch: 100 Loss: 0.000367
[03.04|16:17:35] validation_set Optimizer: 001 Epoch: 100 Loss: 0.010549
[03.04|16:17:36] training_set Optimizer: 002 Epoch: 100 Loss: 0.000246
[03.04|16:17:36] validation_set Optimizer: 002 Epoch: 100 Loss: 0.006286
[03.04|16:17:45] training_set Optimizer: 001 Epoch: 110 Loss: 0.000478
[03.04|16:17:45] validation_set Optimizer: 001 Epoch: 110 Loss: 0.014832
[03.04|16:17:46] training_set Optimizer: 002 Epoch: 110 Loss: 0.000349
[03.04|16:17:46] validation_set Optimizer: 002 Epoch: 110 Loss: 0.007568
[03.04|16:17:57] Execution of optimizer_001.run finished with returncode 0
[03.04|16:17:57] JOB optimizer_001 FINISHED
[03.04|16:17:57] Starting optimizer_001.postrun()
[03.04|16:17:57] optimizer_001.postrun() finished
[03.04|16:17:58] Execution of optimizer_002.run finished with returncode 0
[03.04|16:17:58] JOB optimizer_001 SUCCESSFUL
[03.04|16:17:58] Waiting for job optimizer_002 to finish
[03.04|16:17:58] JOB optimizer_002 FINISHED
[03.04|16:17:58] Starting optimizer_002.postrun()
[03.04|16:17:58] optimizer_002.postrun() finished
[03.04|16:17:58] JOB optimizer_002 SUCCESSFUL
[03.04|16:17:58] PLAMS environment cleaned up successfully
[03.04|16:17:58] PLAMS run finished. Goodbye
[03.04|16:17:59] ParAMSResults
[03.04|16:17:59] Newly created parameter file/dir: step5_attempt11_training/results/optimization/optimizer_001/m3gnet
[03.04|16:17:59] Newly created parameter file/dir: step5_attempt11_training/results/optimization/optimizer_002/m3gnet
[03.04|16:17:59] Done!
[03.04|16:17:59] Deleting step5_attempt10_training
[03.04|16:17:59] ###########################
[03.04|16:17:59] ### Step 5 / Attempt 12 ###
[03.04|16:17:59] ###########################
[03.04|16:17:59] MD Steps: 2280 (cumulative: 3000)
[03.04|16:17:59] Current engine settings:
[03.04|16:17:59]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt11_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt11_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:17:59] Running step5_attempt12_simulation...
[03.04|16:19:06] Job step5_attempt12_simulation finished
[03.04|16:19:06] Deleting files that are no longer needed...
[03.04|16:19:06] Energy uncertainty for final frame of step5_attempt12_simulation: 0.0668 eV
[03.04|16:19:06] 0.0056 eV/atom
[03.04|16:19:06] Forces uncertainty for final frame of step5_attempt12_simulation: 1.0213 eV/angstrom
[03.04|16:19:07] Launching reference calculation
[03.04|16:19:22] Reference calculation finished!
[03.04|16:19:23] Adding results from step5_attempt12_reference_calc1 to training set
[03.04|16:19:23] Current # training set entries: 84
[03.04|16:19:23] Current # validation set entries: 21
[03.04|16:19:23] Storing data in step5_attempt12_reference_data
[03.04|16:19:23] Deleting step5_attempt11_reference_data
[03.04|16:19:23] Deleting step5_attempt12_reference_calc1
[03.04|16:19:23]
[03.04|16:19:23] Current (cumulative) timings:
[03.04|16:19:23] Time (s) Fraction
[03.04|16:19:23] Ref. calcs 947.13 0.145
[03.04|16:19:23] ML training 4194.78 0.641
[03.04|16:19:23] Simulations 1404.04 0.214
[03.04|16:19:23]
[03.04|16:19:23]
[03.04|16:19:23]
[03.04|16:19:23] --- Begin summary ---
[03.04|16:19:23] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:19:23] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:19:23] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:19:23] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:19:23] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:19:23] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:19:23] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:19:23] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:19:23] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:19:23] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:19:23] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:19:23] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:19:23] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:19:23] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:19:23] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:19:23] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:19:23] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:19:23] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:19:23] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:19:23] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:19:23] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:19:23] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:19:23] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:19:23] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:19:23] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:19:23] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:19:23] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:19:23] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:19:23] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:19:23] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:19:23] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:19:23] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:19:23] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:19:23] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:19:23] --- End summary ---
[03.04|16:19:23]
[03.04|16:19:23] Running more reference calculations....
[03.04|16:19:23] Running reference calculations on frames [186, 235] from step5_attempt12_simulation/ams.rkf
[03.04|16:19:23] Calculating 2 frames in total
[03.04|16:19:23] Running step5_attempt12_reference_calc2
[03.04|16:19:38] Running step5_attempt12_reference_calc3
[03.04|16:19:54] Reference calculations finished!
[03.04|16:19:54] Adding results from step5_attempt12_reference_calc2 to training set
[03.04|16:19:54] Adding results from step5_attempt12_reference_calc3 to training set
[03.04|16:19:54] Current # training set entries: 86
[03.04|16:19:54] Current # validation set entries: 21
[03.04|16:19:54] Storing data in step5_attempt12_reference_data
[03.04|16:19:54] Deleting step5_attempt12_reference_calc2
[03.04|16:19:54] Deleting step5_attempt12_reference_calc3
[03.04|16:19:54] Launching reparametrization job: step5_attempt12_training
[03.04|16:19:59] JOB optimizer_001 STARTED
[03.04|16:19:59] Starting optimizer_001.prerun()
[03.04|16:19:59] JOB optimizer_002 STARTED
[03.04|16:19:59] optimizer_001.prerun() finished
[03.04|16:19:59] Starting optimizer_002.prerun()
[03.04|16:19:59] optimizer_002.prerun() finished
[03.04|16:19:59] Waiting for job optimizer_001 to finish
[03.04|16:19:59] JOB optimizer_001 RUNNING
[03.04|16:19:59] JOB optimizer_002 RUNNING
[03.04|16:19:59] Executing optimizer_001.run
[03.04|16:19:59] Executing optimizer_002.run
[03.04|16:21:16] training_set Optimizer: 002 Epoch: 0 Loss: 0.000880
[03.04|16:21:16] validation_set Optimizer: 002 Epoch: 0 Loss: 0.012146
[03.04|16:21:16] training_set Optimizer: 001 Epoch: 0 Loss: 0.001383
[03.04|16:21:16] validation_set Optimizer: 001 Epoch: 0 Loss: 0.016729
[03.04|16:21:28] training_set Optimizer: 002 Epoch: 10 Loss: 0.000306
[03.04|16:21:28] validation_set Optimizer: 002 Epoch: 10 Loss: 0.008971
[03.04|16:21:29] training_set Optimizer: 001 Epoch: 10 Loss: 0.000502
[03.04|16:21:29] validation_set Optimizer: 001 Epoch: 10 Loss: 0.010004
[03.04|16:21:41] training_set Optimizer: 002 Epoch: 20 Loss: 0.000398
[03.04|16:21:41] validation_set Optimizer: 002 Epoch: 20 Loss: 0.017485
[03.04|16:21:41] training_set Optimizer: 001 Epoch: 20 Loss: 0.000455
[03.04|16:21:41] validation_set Optimizer: 001 Epoch: 20 Loss: 0.010039
[03.04|16:21:53] training_set Optimizer: 002 Epoch: 30 Loss: 0.000385
[03.04|16:21:53] validation_set Optimizer: 002 Epoch: 30 Loss: 0.012738
[03.04|16:21:53] training_set Optimizer: 001 Epoch: 30 Loss: 0.000542
[03.04|16:21:53] validation_set Optimizer: 001 Epoch: 30 Loss: 0.014617
[03.04|16:22:05] training_set Optimizer: 002 Epoch: 40 Loss: 0.000357
[03.04|16:22:05] validation_set Optimizer: 002 Epoch: 40 Loss: 0.007508
[03.04|16:22:05] training_set Optimizer: 001 Epoch: 40 Loss: 0.000418
[03.04|16:22:05] validation_set Optimizer: 001 Epoch: 40 Loss: 0.010338
[03.04|16:22:18] training_set Optimizer: 002 Epoch: 50 Loss: 0.000335
[03.04|16:22:18] validation_set Optimizer: 002 Epoch: 50 Loss: 0.006721
[03.04|16:22:18] training_set Optimizer: 001 Epoch: 50 Loss: 0.000376
[03.04|16:22:18] validation_set Optimizer: 001 Epoch: 50 Loss: 0.009940
[03.04|16:22:31] training_set Optimizer: 002 Epoch: 60 Loss: 0.000269
[03.04|16:22:31] validation_set Optimizer: 002 Epoch: 60 Loss: 0.006687
[03.04|16:22:31] training_set Optimizer: 001 Epoch: 60 Loss: 0.000428
[03.04|16:22:31] validation_set Optimizer: 001 Epoch: 60 Loss: 0.012443
[03.04|16:22:43] training_set Optimizer: 002 Epoch: 70 Loss: 0.000286
[03.04|16:22:43] validation_set Optimizer: 002 Epoch: 70 Loss: 0.006047
[03.04|16:22:43] training_set Optimizer: 001 Epoch: 70 Loss: 0.000456
[03.04|16:22:43] validation_set Optimizer: 001 Epoch: 70 Loss: 0.011316
[03.04|16:22:54] training_set Optimizer: 001 Epoch: 80 Loss: 0.000341
[03.04|16:22:54] validation_set Optimizer: 001 Epoch: 80 Loss: 0.010341
[03.04|16:22:54] training_set Optimizer: 002 Epoch: 80 Loss: 0.000293
[03.04|16:22:54] validation_set Optimizer: 002 Epoch: 80 Loss: 0.006811
[03.04|16:23:05] training_set Optimizer: 001 Epoch: 90 Loss: 0.000326
[03.04|16:23:05] validation_set Optimizer: 001 Epoch: 90 Loss: 0.010366
[03.04|16:23:05] training_set Optimizer: 002 Epoch: 90 Loss: 0.000268
[03.04|16:23:05] validation_set Optimizer: 002 Epoch: 90 Loss: 0.013167
[03.04|16:23:16] training_set Optimizer: 001 Epoch: 100 Loss: 0.000292
[03.04|16:23:16] validation_set Optimizer: 001 Epoch: 100 Loss: 0.008577
[03.04|16:23:16] training_set Optimizer: 002 Epoch: 100 Loss: 0.000387
[03.04|16:23:16] validation_set Optimizer: 002 Epoch: 100 Loss: 0.008738
[03.04|16:23:27] training_set Optimizer: 001 Epoch: 110 Loss: 0.000478
[03.04|16:23:27] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011643
[03.04|16:23:27] training_set Optimizer: 002 Epoch: 110 Loss: 0.000246
[03.04|16:23:27] validation_set Optimizer: 002 Epoch: 110 Loss: 0.018030
[03.04|16:23:39] Execution of optimizer_001.run finished with returncode 0
[03.04|16:23:39] JOB optimizer_001 FINISHED
[03.04|16:23:39] Starting optimizer_001.postrun()
[03.04|16:23:39] optimizer_001.postrun() finished
[03.04|16:23:39] JOB optimizer_001 SUCCESSFUL
[03.04|16:23:39] Waiting for job optimizer_002 to finish
[03.04|16:23:40] Execution of optimizer_002.run finished with returncode 0
[03.04|16:23:40] JOB optimizer_002 FINISHED
[03.04|16:23:40] Starting optimizer_002.postrun()
[03.04|16:23:40] optimizer_002.postrun() finished
[03.04|16:23:40] JOB optimizer_002 SUCCESSFUL
[03.04|16:23:40] PLAMS environment cleaned up successfully
[03.04|16:23:40] PLAMS run finished. Goodbye
[03.04|16:23:41] ParAMSResults
[03.04|16:23:41] Newly created parameter file/dir: step5_attempt12_training/results/optimization/optimizer_001/m3gnet
[03.04|16:23:41] Newly created parameter file/dir: step5_attempt12_training/results/optimization/optimizer_002/m3gnet
[03.04|16:23:41] Done!
[03.04|16:23:41] Deleting step5_attempt11_training
[03.04|16:23:41] ###########################
[03.04|16:23:41] ### Step 5 / Attempt 13 ###
[03.04|16:23:41] ###########################
[03.04|16:23:41] MD Steps: 2280 (cumulative: 3000)
[03.04|16:23:41] Current engine settings:
[03.04|16:23:41]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt12_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt12_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:23:41] Running step5_attempt13_simulation...
[03.04|16:24:49] Job step5_attempt13_simulation finished
[03.04|16:24:49] Deleting files that are no longer needed...
[03.04|16:24:49] Energy uncertainty for final frame of step5_attempt13_simulation: 0.1700 eV
[03.04|16:24:49] 0.0142 eV/atom
[03.04|16:24:49] Forces uncertainty for final frame of step5_attempt13_simulation: 1.7373 eV/angstrom
[03.04|16:24:50] Launching reference calculation
[03.04|16:25:04] Reference calculation finished!
[03.04|16:25:04] Adding results from step5_attempt13_reference_calc1 to training set
[03.04|16:25:04] Current # training set entries: 87
[03.04|16:25:04] Current # validation set entries: 21
[03.04|16:25:04] Storing data in step5_attempt13_reference_data
[03.04|16:25:05] Deleting step5_attempt12_reference_data
[03.04|16:25:05] Deleting step5_attempt13_reference_calc1
[03.04|16:25:05]
[03.04|16:25:05] Current (cumulative) timings:
[03.04|16:25:05] Time (s) Fraction
[03.04|16:25:05] Ref. calcs 992.25 0.144
[03.04|16:25:05] ML training 4421.03 0.642
[03.04|16:25:05] Simulations 1472.48 0.214
[03.04|16:25:05]
[03.04|16:25:05]
[03.04|16:25:05]
[03.04|16:25:05] --- Begin summary ---
[03.04|16:25:05] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:25:05] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:25:05] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:25:05] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:25:05] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:25:05] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:25:05] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:25:05] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:25:05] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:25:05] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:25:05] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:25:05] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:25:05] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:25:05] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:25:05] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:25:05] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:25:05] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:25:05] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:25:05] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:25:05] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:25:05] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:25:05] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:25:05] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:25:05] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:25:05] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:25:05] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:25:05] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:25:05] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:25:05] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:25:05] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:25:05] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:25:05] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:25:05] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:25:05] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:25:05] 5 13 FAILED GRADIENTS_UNCERTAINTY 1.7373
[03.04|16:25:05] --- End summary ---
[03.04|16:25:05]
[03.04|16:25:05] Running more reference calculations....
[03.04|16:25:05] Running reference calculations on frames [140, 191] from step5_attempt13_simulation/ams.rkf
[03.04|16:25:05] Calculating 2 frames in total
[03.04|16:25:05] Running step5_attempt13_reference_calc2
[03.04|16:25:19] Running step5_attempt13_reference_calc3
[03.04|16:25:34] Reference calculations finished!
[03.04|16:25:34] Adding results from step5_attempt13_reference_calc2 to training set
[03.04|16:25:34] Adding results from step5_attempt13_reference_calc3 to training set
[03.04|16:25:34] Current # training set entries: 89
[03.04|16:25:34] Current # validation set entries: 21
[03.04|16:25:34] Storing data in step5_attempt13_reference_data
[03.04|16:25:35] Deleting step5_attempt13_reference_calc2
[03.04|16:25:35] Deleting step5_attempt13_reference_calc3
[03.04|16:25:35] Launching reparametrization job: step5_attempt13_training
[03.04|16:25:39] JOB optimizer_001 STARTED
[03.04|16:25:39] JOB optimizer_002 STARTED
[03.04|16:25:39] Starting optimizer_001.prerun()
[03.04|16:25:39] Starting optimizer_002.prerun()
[03.04|16:25:39] optimizer_001.prerun() finished
[03.04|16:25:39] optimizer_002.prerun() finished
[03.04|16:25:40] Waiting for job optimizer_001 to finish
[03.04|16:25:40] JOB optimizer_002 RUNNING
[03.04|16:25:40] JOB optimizer_001 RUNNING
[03.04|16:25:40] Executing optimizer_002.run
[03.04|16:25:40] Executing optimizer_001.run
[03.04|16:26:55] training_set Optimizer: 002 Epoch: 0 Loss: 0.000875
[03.04|16:26:55] validation_set Optimizer: 002 Epoch: 0 Loss: 0.011593
[03.04|16:26:55] training_set Optimizer: 001 Epoch: 0 Loss: 0.002216
[03.04|16:26:55] validation_set Optimizer: 001 Epoch: 0 Loss: 0.020059
[03.04|16:27:06] training_set Optimizer: 002 Epoch: 10 Loss: 0.000277
[03.04|16:27:06] validation_set Optimizer: 002 Epoch: 10 Loss: 0.008927
[03.04|16:27:06] training_set Optimizer: 001 Epoch: 10 Loss: 0.000388
[03.04|16:27:06] validation_set Optimizer: 001 Epoch: 10 Loss: 0.008798
[03.04|16:27:17] training_set Optimizer: 002 Epoch: 20 Loss: 0.000271
[03.04|16:27:17] validation_set Optimizer: 002 Epoch: 20 Loss: 0.012354
[03.04|16:27:17] training_set Optimizer: 001 Epoch: 20 Loss: 0.000361
[03.04|16:27:17] validation_set Optimizer: 001 Epoch: 20 Loss: 0.012050
[03.04|16:27:28] training_set Optimizer: 002 Epoch: 30 Loss: 0.000350
[03.04|16:27:28] validation_set Optimizer: 002 Epoch: 30 Loss: 0.006227
[03.04|16:27:28] training_set Optimizer: 001 Epoch: 30 Loss: 0.000373
[03.04|16:27:28] validation_set Optimizer: 001 Epoch: 30 Loss: 0.011773
[03.04|16:27:39] training_set Optimizer: 002 Epoch: 40 Loss: 0.000266
[03.04|16:27:39] validation_set Optimizer: 002 Epoch: 40 Loss: 0.006298
[03.04|16:27:39] training_set Optimizer: 001 Epoch: 40 Loss: 0.000438
[03.04|16:27:39] validation_set Optimizer: 001 Epoch: 40 Loss: 0.008726
[03.04|16:27:50] training_set Optimizer: 002 Epoch: 50 Loss: 0.000459
[03.04|16:27:50] validation_set Optimizer: 002 Epoch: 50 Loss: 0.011919
[03.04|16:27:50] training_set Optimizer: 001 Epoch: 50 Loss: 0.000423
[03.04|16:27:50] validation_set Optimizer: 001 Epoch: 50 Loss: 0.011780
[03.04|16:28:01] training_set Optimizer: 002 Epoch: 60 Loss: 0.000345
[03.04|16:28:01] validation_set Optimizer: 002 Epoch: 60 Loss: 0.008230
[03.04|16:28:01] training_set Optimizer: 001 Epoch: 60 Loss: 0.000393
[03.04|16:28:01] validation_set Optimizer: 001 Epoch: 60 Loss: 0.010083
[03.04|16:28:13] training_set Optimizer: 002 Epoch: 70 Loss: 0.000292
[03.04|16:28:13] validation_set Optimizer: 002 Epoch: 70 Loss: 0.011302
[03.04|16:28:13] training_set Optimizer: 001 Epoch: 70 Loss: 0.000334
[03.04|16:28:13] validation_set Optimizer: 001 Epoch: 70 Loss: 0.012014
[03.04|16:28:24] training_set Optimizer: 002 Epoch: 80 Loss: 0.000332
[03.04|16:28:24] validation_set Optimizer: 002 Epoch: 80 Loss: 0.007147
[03.04|16:28:25] training_set Optimizer: 001 Epoch: 80 Loss: 0.000313
[03.04|16:28:25] validation_set Optimizer: 001 Epoch: 80 Loss: 0.010347
[03.04|16:28:38] training_set Optimizer: 001 Epoch: 90 Loss: 0.000421
[03.04|16:28:38] validation_set Optimizer: 001 Epoch: 90 Loss: 0.010637
[03.04|16:28:38] training_set Optimizer: 002 Epoch: 90 Loss: 0.000368
[03.04|16:28:38] validation_set Optimizer: 002 Epoch: 90 Loss: 0.006849
[03.04|16:28:50] training_set Optimizer: 002 Epoch: 100 Loss: 0.000224
[03.04|16:28:50] validation_set Optimizer: 002 Epoch: 100 Loss: 0.007558
[03.04|16:28:51] training_set Optimizer: 001 Epoch: 100 Loss: 0.000456
[03.04|16:28:51] validation_set Optimizer: 001 Epoch: 100 Loss: 0.013371
[03.04|16:29:04] training_set Optimizer: 002 Epoch: 110 Loss: 0.000284
[03.04|16:29:04] validation_set Optimizer: 002 Epoch: 110 Loss: 0.009308
[03.04|16:29:04] training_set Optimizer: 001 Epoch: 110 Loss: 0.000335
[03.04|16:29:04] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011433
[03.04|16:29:18] Execution of optimizer_002.run finished with returncode 0
[03.04|16:29:18] JOB optimizer_002 FINISHED
[03.04|16:29:18] Starting optimizer_002.postrun()
[03.04|16:29:18] optimizer_002.postrun() finished
[03.04|16:29:18] JOB optimizer_002 SUCCESSFUL
[03.04|16:29:18] Execution of optimizer_001.run finished with returncode 0
[03.04|16:29:18] JOB optimizer_001 FINISHED
[03.04|16:29:18] Starting optimizer_001.postrun()
[03.04|16:29:18] optimizer_001.postrun() finished
[03.04|16:29:19] JOB optimizer_001 SUCCESSFUL
[03.04|16:29:19] PLAMS environment cleaned up successfully
[03.04|16:29:19] PLAMS run finished. Goodbye
[03.04|16:29:19] ParAMSResults
[03.04|16:29:19] Newly created parameter file/dir: step5_attempt13_training/results/optimization/optimizer_001/m3gnet
[03.04|16:29:19] Newly created parameter file/dir: step5_attempt13_training/results/optimization/optimizer_002/m3gnet
[03.04|16:29:19] Done!
[03.04|16:29:19] Deleting step5_attempt12_training
[03.04|16:29:19] ###########################
[03.04|16:29:19] ### Step 5 / Attempt 14 ###
[03.04|16:29:19] ###########################
[03.04|16:29:19] MD Steps: 2280 (cumulative: 3000)
[03.04|16:29:19] Current engine settings:
[03.04|16:29:19]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt13_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt13_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:29:19] Running step5_attempt14_simulation...
[03.04|16:30:28] Job step5_attempt14_simulation finished
[03.04|16:30:28] Deleting files that are no longer needed...
[03.04|16:30:28] Energy uncertainty for final frame of step5_attempt14_simulation: 0.0486 eV
[03.04|16:30:28] 0.0041 eV/atom
[03.04|16:30:28] Forces uncertainty for final frame of step5_attempt14_simulation: 1.4875 eV/angstrom
[03.04|16:30:28] Launching reference calculation
[03.04|16:30:43] Reference calculation finished!
[03.04|16:30:43] Adding results from step5_attempt14_reference_calc1 to training set
[03.04|16:30:43] Current # training set entries: 90
[03.04|16:30:43] Current # validation set entries: 21
[03.04|16:30:43] Storing data in step5_attempt14_reference_data
[03.04|16:30:43] Deleting step5_attempt13_reference_data
[03.04|16:30:43] Deleting step5_attempt14_reference_calc1
[03.04|16:30:43]
[03.04|16:30:43] Current (cumulative) timings:
[03.04|16:30:43] Time (s) Fraction
[03.04|16:30:43] Ref. calcs 1035.62 0.143
[03.04|16:30:43] ML training 4645.85 0.643
[03.04|16:30:43] Simulations 1540.63 0.213
[03.04|16:30:43]
[03.04|16:30:43]
[03.04|16:30:43]
[03.04|16:30:43] --- Begin summary ---
[03.04|16:30:43] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:30:43] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:30:43] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:30:43] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:30:43] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:30:43] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:30:43] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:30:43] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:30:43] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:30:43] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:30:43] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:30:43] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:30:43] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:30:43] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:30:43] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:30:43] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:30:43] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:30:43] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:30:43] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:30:43] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:30:43] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:30:43] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:30:43] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:30:43] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:30:43] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:30:43] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:30:43] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:30:43] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:30:43] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:30:43] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:30:43] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:30:43] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:30:43] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:30:43] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:30:43] 5 13 FAILED GRADIENTS_UNCERTAINTY 1.7373
[03.04|16:30:43] 5 14 FAILED GRADIENTS_UNCERTAINTY 1.4875
[03.04|16:30:43] --- End summary ---
[03.04|16:30:43]
[03.04|16:30:43] Running more reference calculations....
[03.04|16:30:43] Running reference calculations on frames [139, 188] from step5_attempt14_simulation/ams.rkf
[03.04|16:30:43] Calculating 2 frames in total
[03.04|16:30:43] Running step5_attempt14_reference_calc2
[03.04|16:30:58] Running step5_attempt14_reference_calc3
[03.04|16:31:12] Reference calculations finished!
[03.04|16:31:12] Adding results from step5_attempt14_reference_calc2 to training set
[03.04|16:31:12] Adding results from step5_attempt14_reference_calc3 to training set
[03.04|16:31:12] Current # training set entries: 92
[03.04|16:31:12] Current # validation set entries: 21
[03.04|16:31:12] Storing data in step5_attempt14_reference_data
[03.04|16:31:12] Deleting step5_attempt14_reference_calc2
[03.04|16:31:12] Deleting step5_attempt14_reference_calc3
[03.04|16:31:12] Launching reparametrization job: step5_attempt14_training
[03.04|16:31:17] JOB optimizer_001 STARTED
[03.04|16:31:17] JOB optimizer_002 STARTED
[03.04|16:31:17] Starting optimizer_001.prerun()
[03.04|16:31:17] optimizer_001.prerun() finished
[03.04|16:31:17] Starting optimizer_002.prerun()
[03.04|16:31:17] optimizer_002.prerun() finished
[03.04|16:31:17] JOB optimizer_001 RUNNING
[03.04|16:31:17] JOB optimizer_002 RUNNING
[03.04|16:31:17] Executing optimizer_001.run
[03.04|16:31:17] Executing optimizer_002.run
[03.04|16:31:17] Waiting for job optimizer_001 to finish
[03.04|16:32:32] training_set Optimizer: 002 Epoch: 0 Loss: 0.001041
[03.04|16:32:32] validation_set Optimizer: 002 Epoch: 0 Loss: 0.013440
[03.04|16:32:33] training_set Optimizer: 001 Epoch: 0 Loss: 0.001646
[03.04|16:32:33] validation_set Optimizer: 001 Epoch: 0 Loss: 0.017741
[03.04|16:32:44] training_set Optimizer: 002 Epoch: 10 Loss: 0.000242
[03.04|16:32:44] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009028
[03.04|16:32:45] training_set Optimizer: 001 Epoch: 10 Loss: 0.000324
[03.04|16:32:45] validation_set Optimizer: 001 Epoch: 10 Loss: 0.013727
[03.04|16:32:56] training_set Optimizer: 002 Epoch: 20 Loss: 0.000199
[03.04|16:32:56] validation_set Optimizer: 002 Epoch: 20 Loss: 0.006770
[03.04|16:32:57] training_set Optimizer: 001 Epoch: 20 Loss: 0.000369
[03.04|16:32:57] validation_set Optimizer: 001 Epoch: 20 Loss: 0.009528
[03.04|16:33:08] training_set Optimizer: 002 Epoch: 30 Loss: 0.000410
[03.04|16:33:08] validation_set Optimizer: 002 Epoch: 30 Loss: 0.007665
[03.04|16:33:08] training_set Optimizer: 001 Epoch: 30 Loss: 0.000463
[03.04|16:33:08] validation_set Optimizer: 001 Epoch: 30 Loss: 0.011199
[03.04|16:33:19] training_set Optimizer: 002 Epoch: 40 Loss: 0.000275
[03.04|16:33:19] validation_set Optimizer: 002 Epoch: 40 Loss: 0.008117
[03.04|16:33:20] training_set Optimizer: 001 Epoch: 40 Loss: 0.000382
[03.04|16:33:20] validation_set Optimizer: 001 Epoch: 40 Loss: 0.010930
[03.04|16:33:31] training_set Optimizer: 002 Epoch: 50 Loss: 0.000211
[03.04|16:33:31] validation_set Optimizer: 002 Epoch: 50 Loss: 0.012419
[03.04|16:33:31] training_set Optimizer: 001 Epoch: 50 Loss: 0.000284
[03.04|16:33:31] validation_set Optimizer: 001 Epoch: 50 Loss: 0.008410
[03.04|16:33:34] Execution of optimizer_002.run finished with returncode 0
[03.04|16:33:34] JOB optimizer_002 FINISHED
[03.04|16:33:34] Starting optimizer_002.postrun()
[03.04|16:33:34] optimizer_002.postrun() finished
[03.04|16:33:34] JOB optimizer_002 SUCCESSFUL
[03.04|16:33:41] training_set Optimizer: 001 Epoch: 60 Loss: 0.000296
[03.04|16:33:41] validation_set Optimizer: 001 Epoch: 60 Loss: 0.009538
[03.04|16:33:50] training_set Optimizer: 001 Epoch: 70 Loss: 0.000293
[03.04|16:33:50] validation_set Optimizer: 001 Epoch: 70 Loss: 0.010322
[03.04|16:33:59] training_set Optimizer: 001 Epoch: 80 Loss: 0.000386
[03.04|16:33:59] validation_set Optimizer: 001 Epoch: 80 Loss: 0.009015
[03.04|16:34:09] training_set Optimizer: 001 Epoch: 90 Loss: 0.000296
[03.04|16:34:09] validation_set Optimizer: 001 Epoch: 90 Loss: 0.009336
[03.04|16:34:18] training_set Optimizer: 001 Epoch: 100 Loss: 0.000295
[03.04|16:34:18] validation_set Optimizer: 001 Epoch: 100 Loss: 0.010043
[03.04|16:34:27] training_set Optimizer: 001 Epoch: 110 Loss: 0.000286
[03.04|16:34:27] validation_set Optimizer: 001 Epoch: 110 Loss: 0.008032
[03.04|16:34:38] Execution of optimizer_001.run finished with returncode 0
[03.04|16:34:38] JOB optimizer_001 FINISHED
[03.04|16:34:38] Starting optimizer_001.postrun()
[03.04|16:34:38] optimizer_001.postrun() finished
[03.04|16:34:38] JOB optimizer_001 SUCCESSFUL
[03.04|16:34:38] PLAMS environment cleaned up successfully
[03.04|16:34:38] PLAMS run finished. Goodbye
[03.04|16:34:39] ParAMSResults
[03.04|16:34:39] Newly created parameter file/dir: step5_attempt14_training/results/optimization/optimizer_001/m3gnet
[03.04|16:34:39] Newly created parameter file/dir: step5_attempt14_training/results/optimization/optimizer_002/m3gnet
[03.04|16:34:39] Done!
[03.04|16:34:39] Deleting step5_attempt13_training
[03.04|16:34:39] ###########################
[03.04|16:34:39] ### Step 5 / Attempt 15 ###
[03.04|16:34:39] ###########################
[03.04|16:34:39] MD Steps: 2280 (cumulative: 3000)
[03.04|16:34:39] Current engine settings:
[03.04|16:34:39]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt14_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt14_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:34:39] Running step5_attempt15_simulation...
[03.04|16:35:46] Job step5_attempt15_simulation finished
[03.04|16:35:46] Deleting files that are no longer needed...
[03.04|16:35:46] Energy uncertainty for final frame of step5_attempt15_simulation: 0.0996 eV
[03.04|16:35:46] 0.0083 eV/atom
[03.04|16:35:46] Forces uncertainty for final frame of step5_attempt15_simulation: 1.1512 eV/angstrom
[03.04|16:35:47] Launching reference calculation
[03.04|16:36:01] Reference calculation finished!
[03.04|16:36:02] Adding results from step5_attempt15_reference_calc1 to training set
[03.04|16:36:02] Current # training set entries: 93
[03.04|16:36:02] Current # validation set entries: 21
[03.04|16:36:02] Storing data in step5_attempt15_reference_data
[03.04|16:36:02] Deleting step5_attempt14_reference_data
[03.04|16:36:02] Maximum number of attempts (15) for step 5 reached.
[03.04|16:36:02] Deleting step5_attempt15_reference_calc1
[03.04|16:36:02]
[03.04|16:36:02] Current (cumulative) timings:
[03.04|16:36:02] Time (s) Fraction
[03.04|16:36:02] Ref. calcs 1078.31 0.143
[03.04|16:36:02] ML training 4852.09 0.644
[03.04|16:36:02] Simulations 1608.41 0.213
[03.04|16:36:02]
[03.04|16:36:02]
[03.04|16:36:02]
[03.04|16:36:02] --- Begin summary ---
[03.04|16:36:02] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:36:02] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:36:02] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:36:02] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:36:02] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:36:02] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:36:02] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:36:02] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:36:02] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:36:02] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:36:02] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:36:02] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:36:02] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:36:02] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:36:02] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:36:02] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:36:02] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:36:02] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:36:02] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:36:02] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:36:02] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:36:02] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:36:02] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:36:02] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:36:02] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:36:02] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:36:02] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:36:02] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:36:02] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:36:02] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:36:02] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:36:02] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:36:02] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:36:02] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:36:02] 5 13 FAILED GRADIENTS_UNCERTAINTY 1.7373
[03.04|16:36:02] 5 14 FAILED GRADIENTS_UNCERTAINTY 1.4875
[03.04|16:36:02] 5 15 FAILED GRADIENTS_UNCERTAINTY 1.1512
[03.04|16:36:02] --- End summary ---
[03.04|16:36:02]
[03.04|16:36:02] Running more reference calculations....
[03.04|16:36:02] Running reference calculations on frames [190, 240] from step5_attempt15_simulation/ams.rkf
[03.04|16:36:02] Calculating 2 frames in total
[03.04|16:36:02] Running step5_attempt15_reference_calc2
[03.04|16:36:16] Running step5_attempt15_reference_calc3
[03.04|16:36:31] Reference calculations finished!
[03.04|16:36:31] Adding results from step5_attempt15_reference_calc2 to training set
[03.04|16:36:31] Adding results from step5_attempt15_reference_calc3 to training set
[03.04|16:36:31] Current # training set entries: 95
[03.04|16:36:31] Current # validation set entries: 21
[03.04|16:36:31] Storing data in step5_attempt15_reference_data
[03.04|16:36:32] Maximum number of attempts (15) for step 5 reached.
[03.04|16:36:32] Deleting step5_attempt15_reference_calc2
[03.04|16:36:32] Maximum number of attempts (15) for step 5 reached.
[03.04|16:36:32] Deleting step5_attempt15_reference_calc3
[03.04|16:36:32] Launching reparametrization job: step5_attempt15_training
[03.04|16:36:36] JOB optimizer_001 STARTED
[03.04|16:36:36] JOB optimizer_002 STARTED
[03.04|16:36:36] Starting optimizer_001.prerun()
[03.04|16:36:36] optimizer_001.prerun() finished
[03.04|16:36:36] Starting optimizer_002.prerun()
[03.04|16:36:36] optimizer_002.prerun() finished
[03.04|16:36:37] Waiting for job optimizer_001 to finish
[03.04|16:36:37] JOB optimizer_002 RUNNING
[03.04|16:36:37] Executing optimizer_002.run
[03.04|16:36:37] JOB optimizer_001 RUNNING
[03.04|16:36:37] Executing optimizer_001.run
[03.04|16:37:30] training_set Optimizer: 002 Epoch: 0 Loss: 0.001058
[03.04|16:37:30] validation_set Optimizer: 002 Epoch: 0 Loss: 0.012464
[03.04|16:37:31] training_set Optimizer: 001 Epoch: 0 Loss: 0.001538
[03.04|16:37:31] validation_set Optimizer: 001 Epoch: 0 Loss: 0.011207
[03.04|16:37:41] training_set Optimizer: 002 Epoch: 10 Loss: 0.000252
[03.04|16:37:41] validation_set Optimizer: 002 Epoch: 10 Loss: 0.007358
[03.04|16:37:42] training_set Optimizer: 001 Epoch: 10 Loss: 0.000301
[03.04|16:37:42] validation_set Optimizer: 001 Epoch: 10 Loss: 0.009397
[03.04|16:37:53] training_set Optimizer: 002 Epoch: 20 Loss: 0.000214
[03.04|16:37:53] validation_set Optimizer: 002 Epoch: 20 Loss: 0.009976
[03.04|16:37:54] training_set Optimizer: 001 Epoch: 20 Loss: 0.000325
[03.04|16:37:54] validation_set Optimizer: 001 Epoch: 20 Loss: 0.009197
[03.04|16:38:04] training_set Optimizer: 002 Epoch: 30 Loss: 0.000216
[03.04|16:38:04] validation_set Optimizer: 002 Epoch: 30 Loss: 0.009490
[03.04|16:38:05] training_set Optimizer: 001 Epoch: 30 Loss: 0.000428
[03.04|16:38:05] validation_set Optimizer: 001 Epoch: 30 Loss: 0.009469
[03.04|16:38:16] training_set Optimizer: 002 Epoch: 40 Loss: 0.000213
[03.04|16:38:16] validation_set Optimizer: 002 Epoch: 40 Loss: 0.006003
[03.04|16:38:17] training_set Optimizer: 001 Epoch: 40 Loss: 0.000266
[03.04|16:38:17] validation_set Optimizer: 001 Epoch: 40 Loss: 0.011486
[03.04|16:38:27] training_set Optimizer: 002 Epoch: 50 Loss: 0.000244
[03.04|16:38:27] validation_set Optimizer: 002 Epoch: 50 Loss: 0.007054
[03.04|16:38:29] training_set Optimizer: 001 Epoch: 50 Loss: 0.000385
[03.04|16:38:29] validation_set Optimizer: 001 Epoch: 50 Loss: 0.012710
[03.04|16:38:39] training_set Optimizer: 002 Epoch: 60 Loss: 0.000330
[03.04|16:38:39] validation_set Optimizer: 002 Epoch: 60 Loss: 0.007217
[03.04|16:38:41] training_set Optimizer: 001 Epoch: 60 Loss: 0.000416
[03.04|16:38:41] validation_set Optimizer: 001 Epoch: 60 Loss: 0.011371
[03.04|16:38:53] training_set Optimizer: 002 Epoch: 70 Loss: 0.000351
[03.04|16:38:53] validation_set Optimizer: 002 Epoch: 70 Loss: 0.009133
[03.04|16:38:55] training_set Optimizer: 001 Epoch: 70 Loss: 0.000220
[03.04|16:38:55] validation_set Optimizer: 001 Epoch: 70 Loss: 0.009151
[03.04|16:39:05] training_set Optimizer: 002 Epoch: 80 Loss: 0.000214
[03.04|16:39:05] validation_set Optimizer: 002 Epoch: 80 Loss: 0.007430
[03.04|16:39:07] training_set Optimizer: 001 Epoch: 80 Loss: 0.000262
[03.04|16:39:07] validation_set Optimizer: 001 Epoch: 80 Loss: 0.009118
[03.04|16:39:19] training_set Optimizer: 002 Epoch: 90 Loss: 0.000211
[03.04|16:39:19] validation_set Optimizer: 002 Epoch: 90 Loss: 0.007345
[03.04|16:39:21] training_set Optimizer: 001 Epoch: 90 Loss: 0.000353
[03.04|16:39:21] validation_set Optimizer: 001 Epoch: 90 Loss: 0.008787
[03.04|16:39:32] training_set Optimizer: 002 Epoch: 100 Loss: 0.000286
[03.04|16:39:32] validation_set Optimizer: 002 Epoch: 100 Loss: 0.011590
[03.04|16:39:34] training_set Optimizer: 001 Epoch: 100 Loss: 0.000261
[03.04|16:39:34] validation_set Optimizer: 001 Epoch: 100 Loss: 0.008675
[03.04|16:39:45] training_set Optimizer: 002 Epoch: 110 Loss: 0.000167
[03.04|16:39:45] validation_set Optimizer: 002 Epoch: 110 Loss: 0.008682
[03.04|16:39:47] training_set Optimizer: 001 Epoch: 110 Loss: 0.000291
[03.04|16:39:47] validation_set Optimizer: 001 Epoch: 110 Loss: 0.008512
[03.04|16:39:59] Execution of optimizer_002.run finished with returncode 0
[03.04|16:39:59] JOB optimizer_002 FINISHED
[03.04|16:39:59] Starting optimizer_002.postrun()
[03.04|16:39:59] optimizer_002.postrun() finished
[03.04|16:39:59] JOB optimizer_002 SUCCESSFUL
[03.04|16:40:00] Execution of optimizer_001.run finished with returncode 0
[03.04|16:40:00] JOB optimizer_001 FINISHED
[03.04|16:40:00] Starting optimizer_001.postrun()
[03.04|16:40:00] optimizer_001.postrun() finished
[03.04|16:40:01] JOB optimizer_001 SUCCESSFUL
[03.04|16:40:01] PLAMS environment cleaned up successfully
[03.04|16:40:01] PLAMS run finished. Goodbye
[03.04|16:40:01] ParAMSResults
[03.04|16:40:01] Newly created parameter file/dir: step5_attempt15_training/results/optimization/optimizer_001/m3gnet
[03.04|16:40:01] Newly created parameter file/dir: step5_attempt15_training/results/optimization/optimizer_002/m3gnet
[03.04|16:40:01] Done!
[03.04|16:40:01] Deleting step5_attempt14_training
[03.04|16:40:01] ###########################
[03.04|16:40:01] ### Step 5 / Attempt 16 ###
[03.04|16:40:01] ###########################
[03.04|16:40:01] MD Steps: 2280 (cumulative: 3000)
[03.04|16:40:01] Current engine settings:
[03.04|16:40:01]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt15_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt15_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:40:01] Running step5_attempt16_simulation...
[03.04|16:41:12] Job step5_attempt16_simulation finished
[03.04|16:41:12] Deleting files that are no longer needed...
[03.04|16:41:12] Energy uncertainty for final frame of step5_attempt16_simulation: 0.0815 eV
[03.04|16:41:12] 0.0068 eV/atom
[03.04|16:41:12] Forces uncertainty for final frame of step5_attempt16_simulation: 1.0052 eV/angstrom
[03.04|16:41:13] Launching reference calculation
[03.04|16:41:28] Reference calculation finished!
[03.04|16:41:28] Adding results from step5_attempt16_reference_calc1 to training set
[03.04|16:41:28] Current # training set entries: 96
[03.04|16:41:28] Current # validation set entries: 21
[03.04|16:41:28] Storing data in step5_attempt16_reference_data
[03.04|16:41:28] Deleting step5_attempt15_reference_data
[03.04|16:41:28] Maximum number of attempts (15) for step 5 reached.
[03.04|16:41:28] Deleting step5_attempt16_reference_calc1
[03.04|16:41:28]
[03.04|16:41:28] Current (cumulative) timings:
[03.04|16:41:28] Time (s) Fraction
[03.04|16:41:28] Ref. calcs 1122.44 0.143
[03.04|16:41:28] ML training 5061.81 0.644
[03.04|16:41:28] Simulations 1679.11 0.214
[03.04|16:41:28]
[03.04|16:41:28]
[03.04|16:41:29]
[03.04|16:41:29] --- Begin summary ---
[03.04|16:41:29] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:41:29] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:41:29] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:41:29] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:41:29] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:41:29] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:41:29] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:41:29] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:41:29] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:41:29] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:41:29] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:41:29] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:41:29] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:41:29] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:41:29] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:41:29] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:41:29] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:41:29] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:41:29] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:41:29] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:41:29] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:41:29] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:41:29] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:41:29] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:41:29] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:41:29] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:41:29] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:41:29] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:41:29] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:41:29] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:41:29] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:41:29] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:41:29] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:41:29] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:41:29] 5 13 FAILED GRADIENTS_UNCERTAINTY 1.7373
[03.04|16:41:29] 5 14 FAILED GRADIENTS_UNCERTAINTY 1.4875
[03.04|16:41:29] 5 15 FAILED GRADIENTS_UNCERTAINTY 1.1512
[03.04|16:41:29] 5 16 FAILED GRADIENTS_UNCERTAINTY 1.0052
[03.04|16:41:29] --- End summary ---
[03.04|16:41:29]
[03.04|16:41:29] Running more reference calculations....
[03.04|16:41:29] Running reference calculations on frames [140, 191] from step5_attempt16_simulation/ams.rkf
[03.04|16:41:29] Calculating 2 frames in total
[03.04|16:41:29] Running step5_attempt16_reference_calc2
[03.04|16:41:44] Running step5_attempt16_reference_calc3
[03.04|16:41:59] Reference calculations finished!
[03.04|16:42:00] Adding results from step5_attempt16_reference_calc2 to training set
[03.04|16:42:00] Adding results from step5_attempt16_reference_calc3 to training set
[03.04|16:42:00] Current # training set entries: 98
[03.04|16:42:00] Current # validation set entries: 21
[03.04|16:42:00] Storing data in step5_attempt16_reference_data
[03.04|16:42:00] Maximum number of attempts (15) for step 5 reached.
[03.04|16:42:00] Deleting step5_attempt16_reference_calc2
[03.04|16:42:00] Maximum number of attempts (15) for step 5 reached.
[03.04|16:42:00] Deleting step5_attempt16_reference_calc3
[03.04|16:42:00] Launching reparametrization job: step5_attempt16_training
[03.04|16:42:05] JOB optimizer_001 STARTED
[03.04|16:42:05] JOB optimizer_002 STARTED
[03.04|16:42:05] Starting optimizer_001.prerun()
[03.04|16:42:05] optimizer_001.prerun() finished
[03.04|16:42:05] Starting optimizer_002.prerun()
[03.04|16:42:05] optimizer_002.prerun() finished
[03.04|16:42:05] JOB optimizer_001 RUNNING
[03.04|16:42:05] Executing optimizer_001.run
[03.04|16:42:05] JOB optimizer_002 RUNNING
[03.04|16:42:05] Waiting for job optimizer_001 to finish
[03.04|16:42:05] Executing optimizer_002.run
[03.04|16:44:47] training_set Optimizer: 001 Epoch: 0 Loss: 0.001848
[03.04|16:44:47] validation_set Optimizer: 001 Epoch: 0 Loss: 0.018570
[03.04|16:44:48] training_set Optimizer: 002 Epoch: 0 Loss: 0.001936
[03.04|16:44:48] validation_set Optimizer: 002 Epoch: 0 Loss: 0.021980
[03.04|16:45:04] training_set Optimizer: 001 Epoch: 10 Loss: 0.000513
[03.04|16:45:04] validation_set Optimizer: 001 Epoch: 10 Loss: 0.008778
[03.04|16:45:05] training_set Optimizer: 002 Epoch: 10 Loss: 0.000385
[03.04|16:45:05] validation_set Optimizer: 002 Epoch: 10 Loss: 0.009367
[03.04|16:45:21] training_set Optimizer: 001 Epoch: 20 Loss: 0.000616
[03.04|16:45:21] validation_set Optimizer: 001 Epoch: 20 Loss: 0.008510
[03.04|16:45:22] training_set Optimizer: 002 Epoch: 20 Loss: 0.000414
[03.04|16:45:22] validation_set Optimizer: 002 Epoch: 20 Loss: 0.015129
[03.04|16:45:38] training_set Optimizer: 001 Epoch: 30 Loss: 0.000369
[03.04|16:45:38] validation_set Optimizer: 001 Epoch: 30 Loss: 0.010221
[03.04|16:45:38] training_set Optimizer: 002 Epoch: 30 Loss: 0.000246
[03.04|16:45:38] validation_set Optimizer: 002 Epoch: 30 Loss: 0.009377
[03.04|16:45:55] training_set Optimizer: 001 Epoch: 40 Loss: 0.000336
[03.04|16:45:55] validation_set Optimizer: 001 Epoch: 40 Loss: 0.011091
[03.04|16:45:55] training_set Optimizer: 002 Epoch: 40 Loss: 0.000427
[03.04|16:45:55] validation_set Optimizer: 002 Epoch: 40 Loss: 0.006743
[03.04|16:46:11] training_set Optimizer: 002 Epoch: 50 Loss: 0.000229
[03.04|16:46:11] validation_set Optimizer: 002 Epoch: 50 Loss: 0.007764
[03.04|16:46:12] training_set Optimizer: 001 Epoch: 50 Loss: 0.000345
[03.04|16:46:12] validation_set Optimizer: 001 Epoch: 50 Loss: 0.016195
[03.04|16:46:28] training_set Optimizer: 001 Epoch: 60 Loss: 0.000321
[03.04|16:46:28] validation_set Optimizer: 001 Epoch: 60 Loss: 0.010895
[03.04|16:46:28] training_set Optimizer: 002 Epoch: 60 Loss: 0.000285
[03.04|16:46:28] validation_set Optimizer: 002 Epoch: 60 Loss: 0.012122
[03.04|16:46:45] training_set Optimizer: 001 Epoch: 70 Loss: 0.000319
[03.04|16:46:45] validation_set Optimizer: 001 Epoch: 70 Loss: 0.008863
[03.04|16:46:45] training_set Optimizer: 002 Epoch: 70 Loss: 0.000423
[03.04|16:46:45] validation_set Optimizer: 002 Epoch: 70 Loss: 0.011685
[03.04|16:47:01] training_set Optimizer: 001 Epoch: 80 Loss: 0.000280
[03.04|16:47:01] validation_set Optimizer: 001 Epoch: 80 Loss: 0.010767
[03.04|16:47:02] training_set Optimizer: 002 Epoch: 80 Loss: 0.000273
[03.04|16:47:02] validation_set Optimizer: 002 Epoch: 80 Loss: 0.010993
[03.04|16:47:18] training_set Optimizer: 001 Epoch: 90 Loss: 0.000571
[03.04|16:47:18] validation_set Optimizer: 001 Epoch: 90 Loss: 0.013644
[03.04|16:47:19] training_set Optimizer: 002 Epoch: 90 Loss: 0.000200
[03.04|16:47:19] validation_set Optimizer: 002 Epoch: 90 Loss: 0.007213
[03.04|16:47:35] training_set Optimizer: 001 Epoch: 100 Loss: 0.000291
[03.04|16:47:35] validation_set Optimizer: 001 Epoch: 100 Loss: 0.010198
[03.04|16:47:35] training_set Optimizer: 002 Epoch: 100 Loss: 0.000194
[03.04|16:47:35] validation_set Optimizer: 002 Epoch: 100 Loss: 0.005784
[03.04|16:47:49] training_set Optimizer: 001 Epoch: 110 Loss: 0.000209
[03.04|16:47:49] validation_set Optimizer: 001 Epoch: 110 Loss: 0.011611
[03.04|16:47:49] training_set Optimizer: 002 Epoch: 110 Loss: 0.000187
[03.04|16:47:49] validation_set Optimizer: 002 Epoch: 110 Loss: 0.006409
[03.04|16:48:03] Execution of optimizer_002.run finished with returncode 0
[03.04|16:48:03] Execution of optimizer_001.run finished with returncode 0
[03.04|16:48:03] JOB optimizer_001 FINISHED
[03.04|16:48:03] Starting optimizer_001.postrun()
[03.04|16:48:03] optimizer_001.postrun() finished
[03.04|16:48:03] JOB optimizer_002 FINISHED
[03.04|16:48:03] Starting optimizer_002.postrun()
[03.04|16:48:03] optimizer_002.postrun() finished
[03.04|16:48:03] JOB optimizer_002 SUCCESSFUL
[03.04|16:48:03] JOB optimizer_001 SUCCESSFUL
[03.04|16:48:03] PLAMS environment cleaned up successfully
[03.04|16:48:03] PLAMS run finished. Goodbye
[03.04|16:48:04] ParAMSResults
[03.04|16:48:04] Newly created parameter file/dir: step5_attempt16_training/results/optimization/optimizer_001/m3gnet
[03.04|16:48:04] Newly created parameter file/dir: step5_attempt16_training/results/optimization/optimizer_002/m3gnet
[03.04|16:48:04] Done!
[03.04|16:48:04] Deleting step5_attempt15_training
[03.04|16:48:04] ###########################
[03.04|16:48:04] ### Step 5 / Attempt 17 ###
[03.04|16:48:04] ###########################
[03.04|16:48:04] MD Steps: 2280 (cumulative: 3000)
[03.04|16:48:04] Current engine settings:
[03.04|16:48:04]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt16_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt16_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:48:04] Running step5_attempt17_simulation...
[03.04|16:49:33] Job step5_attempt17_simulation finished
[03.04|16:49:33] Deleting files that are no longer needed...
[03.04|16:49:33] Energy uncertainty for final frame of step5_attempt17_simulation: 0.1640 eV
[03.04|16:49:33] 0.0137 eV/atom
[03.04|16:49:33] Forces uncertainty for final frame of step5_attempt17_simulation: 0.2038 eV/angstrom
[03.04|16:49:33] Launching reference calculation
[03.04|16:49:47] Reference calculation finished!
[03.04|16:49:47] Checking success for step5_attempt17
[03.04|16:50:09] CheckEnergy: Checking energy for MDStep3000, n_atoms = 12
[03.04|16:50:09] CheckEnergy: normalization coefficient = 12
[03.04|16:50:09] CheckEnergy: Actual Threshold
[03.04|16:50:09] CheckEnergy: dE/12 -0.0067 0.2000 OK!
[03.04|16:50:09] CheckEnergy: ddE/12 -0.0072 0.0050 Not OK! (relative to step4_attempt7_simulation:MDStep720)
[03.04|16:50:09]
[03.04|16:50:09] CheckForces: Comparing predicted forces to reference forces (eV/angstrom) on MD snapshot
[03.04|16:50:09] CheckForces: ------------
[03.04|16:50:09] CheckForces: Reference job from step5_attempt17_reference_calc1
[03.04|16:50:09] CheckForces: Prediction job from final frame (MDStep3000) of step5_attempt17_simulation
[03.04|16:50:09] CheckForces: ------------
[03.04|16:50:09] CheckForces: Histogram of forces
[03.04|16:50:09] CheckForces: eV/Ang Ref Pred
[03.04|16:50:09] CheckForces: -2 0 0
[03.04|16:50:09] CheckForces: -1 17 20
[03.04|16:50:09] CheckForces: 0 18 15
[03.04|16:50:09] CheckForces: 1 1 1
[03.04|16:50:09] CheckForces: 2 0 0
[03.04|16:50:09] CheckForces: Threshold for 0 force: 0.50 eV/angstrom
[03.04|16:50:09] CheckForces: Force components with an error exceeding the threshold:
[03.04|16:50:09] CheckForces: Ref Pred Delta Threshold
[03.04|16:50:09] CheckForces: 1.26 1.95 0.69 0.57
[03.04|16:50:09] CheckForces: Maximum deviation: 0.690 eV/angstrom
[03.04|16:50:09] CheckForces: Actual Threshold
[03.04|16:50:09] CheckForces: # > thr. 1 0 Not OK!
[03.04|16:50:09] CheckForces: MAE 0.161 0.30 OK!
[03.04|16:50:09] CheckForces: R^2 0.773 0.40 OK!
[03.04|16:50:09] CheckForces: --------------------
[03.04|16:50:09]
[03.04|16:50:09] Adding results from step5_attempt17_reference_calc1 to training set
[03.04|16:50:09] Current # training set entries: 99
[03.04|16:50:09] Current # validation set entries: 21
[03.04|16:50:09] Storing data in step5_attempt17_reference_data
[03.04|16:50:09] Deleting step5_attempt16_reference_data
[03.04|16:50:09] Maximum number of attempts (15) for step 5 reached.
[03.04|16:50:09] Deleting step5_attempt17_reference_calc1
[03.04|16:50:09]
[03.04|16:50:09] Current (cumulative) timings:
[03.04|16:50:09] Time (s) Fraction
[03.04|16:50:09] Ref. calcs 1167.14 0.140
[03.04|16:50:09] ML training 5425.78 0.649
[03.04|16:50:09] Simulations 1767.76 0.211
[03.04|16:50:09]
[03.04|16:50:09]
[03.04|16:50:10] Step 5 finished successfully!
[03.04|16:50:10]
[03.04|16:50:10] --- Begin summary ---
[03.04|16:50:10] Step Attempt Status Reason final_frame_force_uncertainty finalframe_forces_max_delta
[03.04|16:50:10] 1 1 FAILED Inaccurate 0.4190 2.7044
[03.04|16:50:10] 1 2 FAILED Inaccurate 0.3146 1.4761
[03.04|16:50:10] 1 3 FAILED Inaccurate 0.3262 0.8344
[03.04|16:50:10] 1 4 FAILED Inaccurate 0.2426 0.7091
[03.04|16:50:10] 1 5 FAILED Inaccurate 0.2593 0.6131
[03.04|16:50:10] 1 6 FAILED Inaccurate 0.1288 0.5638
[03.04|16:50:10] 1 7 SUCCESS Accurate 0.2579 0.4236
[03.04|16:50:10] 2 1 FAILED Inaccurate 0.4215 1.5513
[03.04|16:50:10] 2 2 FAILED Inaccurate 0.2573 0.7684
[03.04|16:50:10] 2 3 SUCCESS Accurate 0.2217 0.3520
[03.04|16:50:10] 3 1 FAILED Inaccurate 0.5714 1.7555
[03.04|16:50:10] 3 2 FAILED Inaccurate 0.6386 0.6771
[03.04|16:50:10] 3 3 FAILED Inaccurate 0.1751 0.4264
[03.04|16:50:10] 3 4 SUCCESS Accurate 0.1976 0.2150
[03.04|16:50:10] 4 1 FAILED Inaccurate 0.5658 0.8006
[03.04|16:50:10] 4 2 FAILED Inaccurate 0.1602 0.6446
[03.04|16:50:10] 4 3 FAILED Inaccurate 0.2679 0.5711
[03.04|16:50:10] 4 4 FAILED Inaccurate 0.3434 0.5836
[03.04|16:50:10] 4 5 FAILED Inaccurate 0.6446 0.8304
[03.04|16:50:10] 4 6 FAILED Inaccurate 0.2465 0.2908
[03.04|16:50:10] 4 7 SUCCESS Accurate 0.2864 0.3138
[03.04|16:50:10] 5 1 FAILED GRADIENTS_UNCERTAINTY 1.0061
[03.04|16:50:10] 5 2 FAILED GRADIENTS_UNCERTAINTY 1.0275
[03.04|16:50:10] 5 3 FAILED GRADIENTS_UNCERTAINTY 1.1089
[03.04|16:50:10] 5 4 FAILED GRADIENTS_UNCERTAINTY 1.0169
[03.04|16:50:10] 5 5 FAILED GRADIENTS_UNCERTAINTY 1.0401
[03.04|16:50:10] 5 6 FAILED GRADIENTS_UNCERTAINTY 1.0176
[03.04|16:50:10] 5 7 FAILED GRADIENTS_UNCERTAINTY 1.0315
[03.04|16:50:10] 5 8 FAILED GRADIENTS_UNCERTAINTY 1.0083
[03.04|16:50:10] 5 9 FAILED GRADIENTS_UNCERTAINTY 1.0508
[03.04|16:50:10] 5 10 FAILED GRADIENTS_UNCERTAINTY 1.0253
[03.04|16:50:10] 5 11 FAILED GRADIENTS_UNCERTAINTY 1.0613
[03.04|16:50:10] 5 12 FAILED GRADIENTS_UNCERTAINTY 1.0213
[03.04|16:50:10] 5 13 FAILED GRADIENTS_UNCERTAINTY 1.7373
[03.04|16:50:10] 5 14 FAILED GRADIENTS_UNCERTAINTY 1.4875
[03.04|16:50:10] 5 15 FAILED GRADIENTS_UNCERTAINTY 1.1512
[03.04|16:50:10] 5 16 FAILED GRADIENTS_UNCERTAINTY 1.0052
[03.04|16:50:10] 5 17 SUCCESS MaxAttempts=15 0.2038 0.6897
[03.04|16:50:10] --- End summary ---
[03.04|16:50:10]
[03.04|16:50:10] The engine settings for the final trained ML engine are:
[03.04|16:50:10]
Engine Hybrid
Committee
Enabled Yes
End
Energy
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine1
Term Factor=0.5 Region=* UseCappingAtoms=No EngineID=Engine2
End
Engine MLPotential Engine1
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt16_training/results/optimization/optimizer_001/m3gnet
EndEngine
Engine MLPotential Engine2
Backend M3GNet
MLDistanceUnit angstrom
MLEnergyUnit eV
Model Custom
ParameterDir /path/plams_workdir.002/sal/step5_attempt16_training/results/optimization/optimizer_002/m3gnet
EndEngine
EndEngine
[03.04|16:50:10] Active learning finished!
[03.04|16:50:10] Rerunning the simulation with the final parameters...
[03.04|16:51:52] Copying final_production_simulation/ams.rkf to ams.rkf
[03.04|16:51:52] Goodbye!
[03.04|16:51:53] JOB sal FINISHED
[03.04|16:51:53] JOB sal SUCCESSFUL
<scm.simple_active_learning.plams.simple_active_learning_job.SimpleActiveLearningResults at 0x7fae1d9a56d0>
Above we see that during step 5, several attempts failed with the message GRADIENTS_UNCERTAINTY. It is during step 5 that the actual reaction happens. We do not know exactly at what time the reaction will happen (since the ReactionBoost gradually increases the applied force).
In such a case it is useful to have the GradientsUncertainty reasonable simulation criterion. This will immediately stop the simulation when the uncertainty is too high and follow it with a retraining of the model.
New NEB validation¶
Let’s now evalulate with a second-round NEB and replay.
sal_job.results.get_production_engine_settings() returns the engine
settings in the PLAMS Settings format. Let’s first convert it to a PISA
Engine:
def settings2engine(settings):
temporary_d = drivers.AMS.from_settings(settings)
return temporary_d.Engine
e_new = settings2engine(sal_job.results.get_production_engine_settings())
Let’s now create our own active learning loop for NEB where we run the NEB calculation with our trained potential, replay, add points to training set, retrain, rerun NEB etc.
We need to do this since MD SAL may not exactly sample the minimum energy path.
n_loop = 3
ri = params.ResultsImporter.from_yaml(sal_job.results.get_reference_data_directory())
params_results_dir = sal_job.results.get_params_results_directory()
for i in range(n_loop):
neb = get_neb_job(e_new, f"new_neb{i}")
neb.run()
replay = get_replay_job(neb.results.rkfpath(), name=f"new_replay{i}")
replay.run()
plot_neb_comparison(neb, replay, legend=["retrained", "DFT"])
yaml_dir = f"new_yaml_dir{i}"
ri.add_trajectory_singlepoints(replay.results.rkfpath(), properties=["energy", "forces"])
ri.store(yaml_dir, backup=False)
paramsjob = get_params_job(yaml_dir, load_model=params_results_dir, name=f"new_params{i}")
paramsjob.run()
e_new = settings2engine(paramsjob.results.get_production_engine_settings())
params_results_dir = os.path.abspath(paramsjob.results.path)
[03.04|16:53:38] JOB new_neb0 STARTED
[03.04|16:53:38] JOB new_neb0 RUNNING
[03.04|16:55:00] JOB new_neb0 FINISHED
[03.04|16:55:01] JOB new_neb0 SUCCESSFUL
[03.04|16:55:01] JOB new_replay0 STARTED
[03.04|16:55:01] JOB new_replay0 RUNNING
[03.04|17:00:56] JOB new_replay0 FINISHED
[03.04|17:00:56] JOB new_replay0 SUCCESSFUL
[03.04|17:00:57] JOB new_params0 STARTED
[03.04|17:00:57] JOB new_params0 RUNNING
[03.04|17:07:29] JOB new_params0 FINISHED
[03.04|17:07:29] JOB new_params0 SUCCESSFUL
[03.04|17:07:29] JOB new_neb1 STARTED
[03.04|17:07:29] JOB new_neb1 RUNNING
[03.04|17:08:37] JOB new_neb1 FINISHED
[03.04|17:08:38] JOB new_neb1 SUCCESSFUL
[03.04|17:08:38] JOB new_replay1 STARTED
[03.04|17:08:38] JOB new_replay1 RUNNING
[03.04|17:10:45] JOB new_replay1 FINISHED
[03.04|17:10:45] JOB new_replay1 SUCCESSFUL
[03.04|17:10:46] JOB new_params1 STARTED
[03.04|17:10:46] JOB new_params1 RUNNING
[03.04|17:17:55] JOB new_params1 FINISHED
[03.04|17:17:55] JOB new_params1 SUCCESSFUL
[03.04|17:17:56] JOB new_neb2 STARTED
[03.04|17:17:56] JOB new_neb2 RUNNING
[03.04|17:19:12] JOB new_neb2 FINISHED
[03.04|17:19:13] JOB new_neb2 SUCCESSFUL
[03.04|17:19:13] JOB new_replay2 STARTED
[03.04|17:19:13] JOB new_replay2 RUNNING
[03.04|17:21:25] JOB new_replay2 FINISHED
[03.04|17:21:25] JOB new_replay2 SUCCESSFUL
[03.04|17:21:26] JOB new_params2 STARTED
[03.04|17:21:26] JOB new_params2 RUNNING
[03.04|17:29:30] JOB new_params2 FINISHED
[03.04|17:29:30] JOB new_params2 SUCCESSFUL
Complete Python code¶
#!/usr/bin/env amspython
# coding: utf-8
# ## Initial imports
import scm.plams as plams
from scm.simple_active_learning import SimpleActiveLearningJob
from scm.reactmap.tools import reorder_plams_mol
import matplotlib.pyplot as plt
from scm.input_classes import drivers, engines
from scm.libbase import Units
import scm.params as params
import os
import numpy as np
plams.init()
system1 = plams.from_smiles("CC=C", forcefield="uff") # propene
# add a water molecule *at least* 1 angstrom away from all propene atoms
system1.add_molecule(plams.from_smiles("O"), margin=1)
for at in system1:
at.properties = plams.Settings()
system1.delete_all_bonds()
plams.plot_molecule(system1)
plt.title("System 1: propene + H2O")
system2 = plams.from_smiles("CCCO") # 1-propanol
for at in system2:
at.properties = plams.Settings()
system2.delete_all_bonds()
# reorder atoms in system2 to match the order in system1
# this only takes bond breaking and forming into account, the order is not guaranteed to match exactly for all atoms
system2 = reorder_plams_mol(system1, system2)
# Rotate system2 so that the RMSD with respect to system1 is minimized
system2.align2mol(system1)
plams.plot_molecule(system2)
plt.title("System 2: 1-propanol")
# sanity-check that at least the order of elements is identical
assert list(system1.symbols) == list(system2.symbols), f"Something went wrong!"
# Note that this does not guarantee that the atom order is completely the same.
# For example the order of the hydrogen atoms in the CH3 group might be different.
# This means that we cannot just run NEB directly. So let's first run MD ReactionBoost.
# ## Initial Reaction Boost to get reactant and product
# ### Engine settings
#
# Here we use ``e_up`` to refer to the M3GNet Universal Potential.
#
# For the ADF DFT engine we set an electronic temperature and the OptimizeSpinRound option. This helps with SCF convergence, and can converge the SCF to a different spin state when applicable.
e_up = engines.MLPotential()
e_up.Model = "M3GNet-UP-2022"
e_dft = engines.ADF()
e_dft.XC.GGA = "PBE"
e_dft.XC.Dispersion = "GRIMME3 BJDAMP"
e_dft.Basis.Type = "TZP"
e_dft.Unrestricted = True
e_dft.Occupations = "ElectronicTemperature=300 OptimizeSpinRound=0.05"
def set_reaction_boost(driver, nsteps=3000):
driver.Task = "MolecularDynamics"
md = driver.MolecularDynamics
md.InitialVelocities.Temperature = 100
md.NSteps = nsteps
md.ReactionBoost.Type = "Pair"
md.ReactionBoost.BondBreakingRestraints.Type = "Erf"
md.ReactionBoost.BondBreakingRestraints.Erf.MaxForce = 0.05
md.ReactionBoost.BondMakingRestraints.Type = "Erf"
md.ReactionBoost.BondMakingRestraints.Erf.MaxForce = 0.12
md.ReactionBoost.InitialFraction = 0.05
md.ReactionBoost.Change = "LogForce"
md.ReactionBoost.NSteps = nsteps
md.ReactionBoost.TargetSystem[0] = "final"
md.Trajectory.SamplingFreq = 10
md.Trajectory.WriteBonds = False
md.Trajectory.WriteMolecules = False
md.TimeStep = 0.25
md.Thermostat[0].Tau = 5
md.Thermostat[0].Temperature = [100.0]
md.Thermostat[0].Type = "Berendsen"
def get_reaction_boost_job(engine, molecule, name: str = "reaction_boost") -> plams.AMSJob:
d = drivers.AMS()
set_reaction_boost(d)
d.Engine = engine
job = plams.AMSJob(settings=d, name=name, molecule=molecule)
job.settings.runscript.nproc = 1
return job
molecule_dict = {"": system1, "final": system2}
prelim_job = get_reaction_boost_job(e_up, molecule_dict, "prelim_md")
prelim_job.run()
# Let's check that the final molecule corresponds to the target system (1-propanol):
#
engine_energies = prelim_job.results.get_history_property("EngineEnergy")
N_frames = len(engine_energies)
max_index = np.argmax(engine_energies)
reactant_index = max(0, max_index - 50) # zero-based
system1_correct_order = prelim_job.results.get_history_molecule(reactant_index + 1)
system1_correct_order.delete_all_bonds()
plams.plot_molecule(system1_correct_order)
product_index = min(max_index + 50, N_frames - 1) # zero-based
system2_correct_order = prelim_job.results.get_history_molecule(product_index + 1)
system1_correct_order.delete_all_bonds()
plams.plot_molecule(system2_correct_order)
# We now have the product molecule with the correct atom order, which means we can run an initial NEB with M3GNet and compare to the DFT reference:
# ## Initial NEB calculation
molecule_dict = {"": system1_correct_order, "final": system2_correct_order}
def get_neb_job(engine, name: str = "neb") -> plams.AMSJob:
d = drivers.AMS()
d.Task = "NEB"
d.GeometryOptimization.Convergence.Quality = "Basic"
d.NEB.Images = 12
d.Engine = engine
neb_job = plams.AMSJob(name=name, settings=d, molecule=molecule_dict)
return neb_job
neb_job = get_neb_job(e_up, name="neb_up")
neb_job.run()
# Let's then replay with the ADF DFT engine.
def get_replay_job(rkf, name="replay"):
d_replay = drivers.AMS()
d_replay.Task = "Replay"
d_replay.Replay.File = os.path.abspath(rkf)
d_replay.Properties.Gradients = True
d_replay.Engine = e_dft
replay_job = plams.AMSJob(name=name, settings=d_replay)
return replay_job
replay_job = get_replay_job(neb_job.results.rkfpath(), "replay_neb")
replay_job.run()
def get_relative_energies(neb_job):
e = neb_job.results.get_neb_results()["Energies"]
e = np.array(e) - np.min(e)
e *= Units.get_factor("hartree", "eV")
return e
def plot_neb_comparison(neb_job, replay_job, legend=None, title=None):
energies_up = get_relative_energies(neb_job)
energies_dft = get_relative_energies(replay_job)
fig, ax = plt.subplots()
ax.plot(energies_up)
ax.plot(energies_dft)
ax.legend(legend or ["M3GNet-UP-2022", "DFT singlepoints"])
ax.set_ylabel("Relative energy (eV)")
ax.set_title(title or "Reaction path water+propene -> 1-propanol")
return ax
plot_neb_comparison(neb_job, replay_job)
# So we can see that either M3GNet-UP-2022 underestimates the barrier or it NEB path is different from the DFT one. Let's use these datapoints as a starting point for the active learning.
#
# Let's also run replay on some of the frames from the prelim_md reaction boost job:
replay_md = get_replay_job(prelim_job.results.rkfpath(), "replay_md")
N_frames_to_replay = 10
replay_md.settings.input.Replay.Frames = list(
np.linspace(reactant_index, product_index, N_frames_to_replay, dtype=np.int64)
)
replay_md.run()
# ## Simple Active Learning using Uncertainties
# ## Create the initial reference data
yaml_dir = "my_neb_data"
ri = params.ResultsImporter.from_ase() # use ASE units
ri.add_trajectory_singlepoints(replay_job.results.rkfpath(), properties=["energy", "forces"], data_set="training_set")
ri.add_trajectory_singlepoints(
replay_md.results.rkfpath(),
properties=["energy", "forces"],
data_set="training_set",
indices=list(range(1, N_frames_to_replay - 1)),
)
ri.add_trajectory_singlepoints(
replay_md.results.rkfpath(),
properties=["energy", "forces"],
indices=[0, N_frames_to_replay - 1],
data_set="validation_set",
)
ri.store(yaml_dir, backup=False)
# When we have initial reference data like this, it's often most convenient to run a separate ParAMS training before starting the active learning.
#
# This lets us sanity-check the training parameters, and more easily try different Active Learning settings without having to retrain the initial model every time.
def get_params_job(yaml_dir, load_model=None, name="paramsjob"):
committee_size = 2
paramsjob = params.ParAMSJob.from_yaml(yaml_dir, use_relative_paths=True, name=name)
paramsjob.settings.input.Task = "MachineLearning"
ml = paramsjob.settings.input.MachineLearning
ml.Backend = "M3GNet"
if load_model:
ml.LoadModel = load_model
ml.MaxEpochs = 200
ml.M3GNet.LearningRate = 5e-4
else:
ml.M3GNet.Model = "Custom"
ml.M3GNet.Custom.NumNeurons = 32
ml.MaxEpochs = 300
ml.M3GNet.LearningRate = 1e-3
ml.CommitteeSize = committee_size
paramsjob.settings.input.ParallelLevels.CommitteeMembers = committee_size
return paramsjob
paramsjob = get_params_job(yaml_dir, name="custom_initial_training")
paramsjob.run()
# ## Set up the active learning job
#
# Here the key new setting is the ``ReasonableSimulationCriteria.GradientsUncertainty``. This setting will cause the MD simulation to instantly stop if the uncertainty is greater than 1.0 eV/angstrom.
#
# This is useful since the ML model is unlikely to give good predictions for the new types of structures encountered during the reactive MD.
#
# In the summary log, such an event will be marked as "FAILED" with the reason "GRADIENTS_UNCERTAINTY".
#
# In order to use ML uncertainties, you need to train a committee model with at least 2 members. Here we set the commmittee size to 2. We also choose to train the 2 committee members in parallel. By default, they would be trained in sequence.
#
# It is a good idea to train them in parallel if you have the computational resources to do so (for example, enough GPU memory).
#
# When using uncertainty-based critiera, you may consider increasing the MaxAttemptsPerStep. Here, we stick with the default value of 15.
d_al = drivers.SimpleActiveLearning()
d_al.ActiveLearning.InitialReferenceData.Load.FromPreviousModel = True
d_al.ActiveLearning.Steps.Type = "Geometric"
d_al.ActiveLearning.Steps.Geometric.NumSteps = 5
d_al.ActiveLearning.Steps.Geometric.Start = 10
d_al.ActiveLearning.ReasonableSimulationCriteria.GradientsUncertainty.Enabled = True
d_al.ActiveLearning.ReasonableSimulationCriteria.GradientsUncertainty.MaxValue = 1.0 # eV/ang
d_al.ActiveLearning.SuccessCriteria.Forces.MinR2 = 0.4
d_al.ActiveLearning.MaxReferenceCalculationsPerAttempt = 3
d_al.ActiveLearning.MaxAttemptsPerStep = 15
d_al.MachineLearning.Backend = "M3GNet"
d_al.MachineLearning.LoadModel = os.path.abspath(paramsjob.results.path)
d_al.MachineLearning.CommitteeSize = 2
d_al.MachineLearning.MaxEpochs = 120
d_al.MachineLearning.M3GNet.LearningRate = 5e-4
d_al.MachineLearning.RunAMSAtEnd = False
d_al.ParallelLevels.CommitteeMembers = 2
set_reaction_boost(d_al)
d_al.Engine = e_dft
sal_job = SimpleActiveLearningJob(name="sal", driver=d_al, molecule=molecule_dict)
print(sal_job.get_input())
sal_job.run(watch=True)
# Above we see that during step 5, several attempts failed with the message GRADIENTS_UNCERTAINTY. It is during step 5 that the actual reaction happens. We do not know exactly at what time the reaction will happen (since the ReactionBoost gradually increases the applied force).
#
# In such a case it is useful to have the GradientsUncertainty reasonable simulation criterion. This will immediately stop the simulation when the uncertainty is too high and follow it with a retraining of the model.
# ## New NEB validation
# Let's now evalulate with a second-round NEB and replay. ``sal_job.results.get_production_engine_settings()`` returns the engine settings in the PLAMS Settings format. Let's first convert it to a PISA Engine:
def settings2engine(settings):
temporary_d = drivers.AMS.from_settings(settings)
return temporary_d.Engine
e_new = settings2engine(sal_job.results.get_production_engine_settings())
# Let's now create our own active learning loop for NEB where we run the NEB calculation with our trained potential, replay, add points to training set, retrain, rerun NEB etc.
#
# We need to do this since MD SAL may not exactly sample the minimum energy path.
n_loop = 3
ri = params.ResultsImporter.from_yaml(sal_job.results.get_reference_data_directory())
params_results_dir = sal_job.results.get_params_results_directory()
for i in range(n_loop):
neb = get_neb_job(e_new, f"new_neb{i}")
neb.run()
replay = get_replay_job(neb.results.rkfpath(), name=f"new_replay{i}")
replay.run()
plot_neb_comparison(neb, replay, legend=["retrained", "DFT"])
yaml_dir = f"new_yaml_dir{i}"
ri.add_trajectory_singlepoints(replay.results.rkfpath(), properties=["energy", "forces"])
ri.store(yaml_dir, backup=False)
paramsjob = get_params_job(yaml_dir, load_model=params_results_dir, name=f"new_params{i}")
paramsjob.run()
e_new = settings2engine(paramsjob.results.get_production_engine_settings())
params_results_dir = os.path.abspath(paramsjob.results.path)