Utopia features
Data management
Utopia comes with its own object-oriented C++ HDF5 library which takes care of storing the model state into an efficient, open, and frequently-used data format.
For storing data of models with a non-adaptive state size, Utopia aims to makes the data-writing interface as
simple as possible.
However, adaptively-sized models require more elaborate data writing procedures; the DataManager aims to cover
this area and provide a high flexibility.
A model generates data that almost always needs to be stored efficiently.
Utopia comes with a simple-to-use data writing functionality.
You specify what data to write in the write_data function.
When running a simulation, you can easily define when to start writing with what step size via configuration settings.
The model base class automatically translates these settings into functionality.
Utopia provides optional convenience functions that create datasets specialized for CellManager or AgentManager data.
These are suited for models where the number of CA cells or agents does not change.
For writing data of models with varying state size (e.g. due to the number of agents changing adaptively), we advise to set up the DataManager (see below).
Model simulations can easily produce vast amounts of heterogeneous, highly-structured data. Typically, we must decide when to write which kind of data into what dataset. Ideally, we want this to happen in a flexible, easily configurable way.
The DataManager provides a high-level interface to cope with these issues.
It builds datasets in different data groups and triggers writing tasks for each task individually.
For example, it allows you to write some averaged quantity each iteration while others only within predefined low-resolution intervals, perhaps with a few high-resolution windows. Flexibility is key.
Once integrated into a model, you can configure a writing task via the model configuration.
To prevent undesired run-time overhead from the high-level interface, we built the DataManager hugely reliant on compile-time algorithms.
HDF5 is a widely-used data storage and organization format designed for large amounts of heterogeneous data used in the earth sciences, genetics, astronomy, physics, finances, etc.
Utopia implements a low-level modern C++ interface to the HDF5 C-library compatible with the Standard Template Library (STL). It does not wrap the complete C-library but only a feature selection useful and needed for the Utopia project.