Utopia features
Simulation Control and Configuration
Setting up and performing simulations is a central part of the workflow when investigating computer models: it is the interface between the person investigating the model and the model itself.
Utopia provides powerful tools to configure simulations and carry them out: With a versatile configuration system and easy parallelization mechanisms, it aims to make this process as convenient as possible, while maintaining a high degree of flexibility and ensuring reproducibility.
Throughout Utopia, configuration files are used to define how the model and all parts of the framework should behave. The main idea is: everything should be configurable … but there should be good defaults. In the simplest case, one would not need to specify any parameters; but if more control is desired, it is possible to adjust (almost) any parameter.
To achieve this goal, Utopia employs a set of YAML configuration files and an update procedure. Inspired by hierarchical organization and modularization observable in complex systems, Utopia allows to specify default values at several stages: there are framework-level defaults as well as machine-specific and model-specific defaults values.
On top of that, a user may update all these values with a dedicated configuration file for the simulation run that is to be carried out, which may look like this:
# Run configuration for the ForestFire model
---
parameter_space:
num_steps: 10000
write_every: 10
seed: 100
ForestFire:
space:
periodic: false
p_lightning: 3.e-4
The product of this update procedure is the so-called meta-configuration, which contains all information for performing the simulation. By storing the meta-configuration alongside a simulation, reproducing old simulations becomes as easy as feeding the stored meta-configuration to the new simulation.
The Utopia command line interface (CLI) provides control over running and evaluating model simulations. It looks like this:
utopia run ForestFire --num-steps 10k
The above command will run the ForestFire model with default parameters and ten thousand iteration steps.
It will then invoke the data processing pipeline to generate the configured default plots.
For subsequent data evaluation on finished simulations, the utopia eval command gives control over the processing pipeline.
The CLI aims to give access to the most frequently used development tools, for instance: running a model in debug mode or changing certain parameters for the specific run. In a more complex scenario, the CLI is used to specify which YAML configuration file should be used for the simulation run and its evaluation.
Running one and the same model with different parameters is one of the most frequent tasks when studying a model’s behavior. Utopia aims to make this part of the model workflow as easy and frictionless as possible.
By embedding the paramspace python package and with the power of custom YAML tags, defining a parameter sweep in Utopia is as simple as this:
parameter_space:
num_steps: 1000
seed: !sweep
default: 42
range: [10]
ForestFire:
p_lightning: !sweep
default: 1.e-4
values:
- 1.e-3
- 1.e-4
- 1.e-5
The above configuration defines a simulation run with ten different values for the random number generator seed and three different values for the lightning probability of the ForestFire model.
In total, 30 individual simulations will be carried out, covering all combinations of these parameter values.
As all parameters are defined via the YAML syntax, every parameter can be made into a sweep dimension, simply by adding the !sweep tag to the entry in the configuration and specifying the values over which to sweep.
In Utopia, simulations for different points in parameter space run in parallel, making use of available computing power.
As simulations for separate points in parameter space are fully independent from each other, this simple form of parallelization is very efficient and does not require any optimization by a model developer (unlike model-level parallelization).
The WorkerManager of the Utopia frontend provides ample control over how many CPU cores are to be used.
Utopia goes one step further and elevates this approach to cluster architectures: With cluster support, a parameter space can be scanned using distributed computing nodes; the Utopia frontend coordinates which node is to perform simulations for which set of points in parameter space. It is pre-configured for the widely used SLURM Workload Manager but can be easily adapted for other scheduling systems.
In Utopia, a running model simulation may periodically communicate its current state to the frontend.
The information may include the simulation’s progress but also model-specific information; for instance, the ContDisease model provides the individual state densities.
With this information, the Utopia frontend can be used for monitoring how a simulation proceeds, which becomes particularly useful for longer-running simulations which may have ended up in an uninteresting state.
Whether a simulation should continue or not can be automated with so-called stop conditions:
If a monitored value exceeds the valid bounds, the simulation is automatically stopped.
For example, the following configuration stops a simulation after 100 seconds or if the number_of_agents dropped to zero:
run_kwargs:
# Stop after 100s or if the number of agents drops to zero
stop_conditions:
- !stop-condition
func: timeout_wall
seconds: 100
- !stop-condition
name: no_more_agents
description: stops simulation when there are no more agents
func: check_monitor_entry
entry_name: MyModel.number_of_agents
operator: "<="
value: 0
With this feature, computational load can be reduced by letting simulations run only as long as necessary. The configuration-based interface makes it easy to define these conditions and does not require to touch the model implementation.
Running and evaluating simulations with Utopia is not restricted to the command line interface. For uses cases like exploring data output, it is more convenient to do this in an interactive fashion via an IPython session or a Jupyter Notebook.
To that end, the Utopia frontend exposes all the functionality of performing and evaluating simulations:
import utopya
# Instantiate the model
ffm = utopya.Model(name='ForestFire')
# Perform a simulation using default parameters and 100 iteration steps
mv = ffm.create_mv(parameter_space=dict(num_steps=100))
mv.run()
# Load and inspect the data
dm = mv.dm
dm.load_from_cfg(print_tree=True)
# Iterate over all available simulations ("universes")
for uni in dm['multiverse'].values():
# Get this universe's configuration and some data
cfg = uni['cfg']
tree_density = uni['data/ForestFire/tree_density']
# Do something with the data
# ...
Simulations that are run via the Python interface will still write all output to the regular output directory, making them as reproducible as runs carried out via the CLI.