armi.bookkeeping.db.database3 module

ARMI Database implementation, version 3.

This Implementation of the database is a significant departure from the previous. One of the foundational concepts in this version is that a reactor model should be fully recoverable from the database itself; all the way down to the component level. As a result, the structure of the underlying data is bound to the hierarchical Composite Reactor Model, rather than an ad hoc collection of Block parameter fields and other parameters. Furthermore, this format is intended to be more dynamic, permitting as-yet undeveloped levels and classes in the Composite Reactor Model to be supported as they are added. More high-level discussion is contained in The Database File.

The most important contents of this module are the DatabaseInterface, the Database3 class, the Layout class, and the special data packing/unpacking functions. The Database3 class contains most of the functionality for interacting with the underlying data. This includes things like dumping a Reactor state to the database and loading it back again, as well as extracting historical data for a given object or collection of object from the database file. When interacting with the database file, the Layout class is used to help map the hierarchical Composite Reactor Model to the flat representation in the database.

armi.bookkeeping.db.database3.getH5GroupName(cycle, timeNode, statePointName=None)

armi.bookkeeping.db.database3.describeInterfaces(cs)

Function for exposing interface(s) to other code

class armi.bookkeeping.db.database3.DatabaseInterface(r, cs)

Bases: armi.interfaces.Interface

Handles interactions between the ARMI data model and the persistent data storage system.

This reads/writes the ARMI state to/from the database and helps derive state information that can be derived.

Construct an interface.

The r and cs arguments are required, but may be None, where appropriate for the specific Interface implementation.

Parameters

• r (Reactor) – A reactor to attach to

• cs (Settings) – Settings object to use

Raises

RuntimeError – Interfaces derived from Interface must define their name

name = 'database'

property database

Presents the internal database object, if it exists.

interactBOL()

Initialize the database if the main interface was not available.

initDB()

Open the underlying database to be written to, and write input files to DB.

Notes

Main Interface calls this so that the database is available as early as possible in the run. The database interface interacts near the end of the interface stack (so that all the parameters have been updated) while the Main Interface interacts first.

interactEveryNode(cycle, node)

Write to database.

DBs should receive the state information of the run at each node.

interactEOC(cycle=None)

In case anything changed since last cycle (e.g. rxSwing), update DB.

interactEOL()

DB’s should be closed at run’s end.

interactError()

Get shutdown state information even if the run encounters an error

distributable()

Return true if this can be MPI broadcast.

Notes

Cases where this isn’t possible include the database interface, where the SQL driver cannot be distributed.

prepRestartRun(dbCycle, dbNode)

Load the data history from the database being restarted from.

loadState(cycle, timeNode, timeStepName='', fileName=None, updateGlobalAssemNum=True)

Loads a fresh reactor and applies it to the Operator.

Notes

timeStepName is not currently supportred by database load. Will load preferentially from the fileName if passed. Otherwise will load from existing database in memory or cs[“reloadDBName”] in that order.

Raises

RuntimeError – If fileName is specified and that file does not have the time step. If fileName is not specified and neither the database in memory, nor the cs[“reloadDBName”] have the time step specified.

getHistory(comp: armi.reactor.composites.ArmiObject, params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get historical parameter values for a single object.

This is mostly a wrapper around the same function on the Database3 class, but knows how to return the current value as well.

See also

Database3.getHistory()

getHistories(comps: Sequence[armi.reactor.composites.ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None) → Dict[armi.reactor.composites.ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get historical parameter values for one or more objects.

This is mostly a wrapper around the same fumction on the Database3 class, but knows how to return the current value as well.

See also

Database3.getHistories()

class armi.bookkeeping.db.database3.Database3(fileName: str, permission: str)

Bases: armi.bookkeeping.db.database.Database

Version 3 of the ARMI Database, handling serialization and loading of Reactor states.

This implementation of the database pushes all objects in the Composite Reactor Model into the database. This process is aided by the Layout class, which handles the packing and unpacking of the structure of the objects, their relationships, and their non-parameter attributes.

See also

doc/user/outputs/database for more details.

Create a new Database3 object.

Parameters

• fileName – name of the file

• permission – file permissions, write (“w”) or read (“r”)

version = '3'

timeNodeGroupPattern = re.compile('^c(\\d\\d)n(\\d\\d)$')

open()

close(completedSuccessfully=False)

Close the DB and perform cleanups and auto-conversions.

Notes

Visualization tools like XTVIEW are still using DB format 2, so this will (for now) auto-generate a full XTVIEW-compatible database upon closing. It does this here because, in remote runs, this is where the I/O is fastest.

splitDatabase(keepTimeSteps: Sequence[Tuple[int, int]], label: str) → str

Discard all data except for specific time steps, retaining old data in a separate file.

This is useful when performing more exotic analyses, where each “time step” may not represent a specific point in time, but something more nuanced. For example, equilibrium cases store a new “cycle” for each iteration as it attempts to converge the equilibrium cycle. At the end of the run, the last “cycle” is the converged equilibrium cycle, whereas the previous cycles constitute the path to convergence, which we typically wish to discard before further analysis.

Parameters

• keepTimeSteps – A collection of the time steps to retain

• label – An informative label for the backed-up database. Usually something like “-all-iterations”. Will be interposed between the source name and the “.h5” extension.

Returns

The name of the new, backed-up database file.

Return type

str

property fileName

loadCS()

loadBlueprints()

loadGeometry()

writeInputsToDB(cs, csString=None, geomString=None, bpString=None)

Write inputs into the database based the CaseSettings.

This is not DRY on purpose. The goal is that any particular Database implementation should be very stable, so we dont want it to be easy to change one Database implementation’s behavior when trying to change another’s.

Notes

This is hard-coded to read the entire file contents into memory and write that directly into the database. We could have the cs/blueprints/geom write to a string, however the ARMI log file contains a hash of each files’ contents. In the future, we will be able to reproduce calculation showing that the inputs are identical.

readInputsFromDB()

mergeHistory(inputDB, startCycle, startNode)

Copy time step data up to, but not including the passed cycle and node.

Notes

This is used for restart runs with the standard operator for example. The current time step (being loaded from) should not be copied, as that time steps data will be written at the end of the time step.

__enter__()

Context management support

__exit__(type, value, traceback)

Typically we don’t care why it broke but we want the DB to close

genTimeStepGroups(timeSteps: Sequence[Tuple[int, int]] = None) → Generator[h5py._hl.group.Group, None, None]

Returns a generator of HDF5 Groups for all time nodes, or for the passed selection.

genTimeSteps() → Generator[Tuple[int, int], None, None]

Returns a generator of (cycle, node) tuples that are present in the DB.

genAuxiliaryData(ts: Tuple[int, int]) → Generator[str, None, None]

Returns a generator of names of auxiliary data on the requested time point.

getAuxiliaryDataPath(ts: Tuple[int, int], name: str) → str

Get a string describing a path to an auxiliary data location.

Parameters

• ts – The time step that the auxiliary data belongs to

• name – The name of the auxiliary data

Returns

An absolute location for storing auxiliary data with the given name for the given time step

Return type

str

keys()

getH5Group(r, statePointName=None)

Get the H5Group for the current ARMI timestep.

This method can be used to allow other interfaces to place data into the database at the correct timestep.

hasTimeStep(cycle, timeNode, statePointName='')

Returns True if (cycle, timeNode, statePointName) is contained in the database.

writeToDB(reactor, statePointName=None)

load(cycle, node, cs=None, bp=None, geom=None)

Load a new reactor from (cycle, node).

Case settings, blueprints, and geom can be provided by the client, or read from the database itself. Providing these from the client could be useful when performing snapshot runs or the like, where it is expected to use results from a run using different settings, then continue with new settings. Even in this case, the blueprints and geom should probably be the same as the original run.

Parameters

• cycle (int) – cycle number

• node (int) – time node

• cs (armi.settings.Settings (optional)) – if not provided one is read from the database

• bp (armi.reactor.Blueprints (Optional)) – if not provided one is read from the database

• geom (armi.geometry.Geometry (Optional)) – if not provided one is read from the database

getHistory(comp: armi.reactor.composites.ArmiObject, params: Sequence[str] = None, timeSteps: Sequence[Tuple[int, int]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get parameter history for a single ARMI Object.

Parameters

• comps – An individual ArmiObject

• params – parameters to gather

Returns

Dictionary of str/list pairs.

Return type

dict

getHistories(comps: Sequence[armi.reactor.composites.ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[armi.reactor.composites.ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get the parameter histories for a sequence of ARMI Objects.

This implementation is unaware of the state of the reactor outside of the database itself, and is therefore not usually what client code should be calling directly. It only knows about historical data that actually exists in the database. Usually one wants to be able to get historical, plus current data, for which the similar method on the DatabaseInterface is more useful.

Parameters

• comps – Something that is iterable multiple times

• params – parameters to gather.

• timeSteps – Selection of time nodes to get data for. If omitted, return full history

Returns

Dictionary ArmiObject (input): dict of str/list pairs containing ((cycle, node), value).

Return type

dict

class armi.bookkeeping.db.database3.Layout(h5group=None, comp=None)

Bases: object

The Layout class describes the hierarchical layout of the composite structure in a flat representation.

A Layout is built up by starting at the root of a composite tree and recursively appending each node in the tree to the list of data. So for a typical Reactor model, the data will be ordered something like [r, c, a1, a1b1, a1b1c1, a1b1c2, a1b2, a1b2c1, …, a2, …]

The layout is also responsible for storing Component attributes, like location, material, and temperatures (from blueprints), which aren’t stored as Parameters. Temperatures, specifically, are rather complicated beasts in ARMI, and more fundamental changes to how we deal with them may allow us to remove them from Layout.

writeToDB(h5group)

armi.bookkeeping.db.database3.allSubclasses(cls)

This currently include Materials… and it should not.

armi.bookkeeping.db.database3.packSpecialData(data: numpy.ndarray, paramName: str) → Tuple[Optional[numpy.ndarray], Dict[str, Any]]

Reduce data that wouldn’t otherwise play nicely with HDF5/numpy arrays to a format that will.

This is the main entry point for conforming “strange” data into something that will both fit into a numpy array/HDF5 dataset, and be recoverable to its original-ish state when reading it back in. This is accomplished by detecting a handful of known offenders and using various HDF5 attributes to store necessary auxiliary data. It is important to keep in mind that the data that is passed in has already been converted to a numpy array, so the top dimension is always representing the collection of composites that are storing the parameters. For instance, if we are dealing with a Block parameter, the first index in the numpy array of data is the block index; so if each block has a parameter that is a dictionary, data would be a ndarray, where each element is a dictionary. This routine supports a number of different “strange” things: * Dict[str, float]: These are stored by finding the set of all keys for all

instances, and storing those keys as a list in an attribute. The data themselves are stored as arrays indexed by object, then key index. Dictionaries lacking data for a key store a nan in it’s place. This will work well in instances where most objects have data for most keys.

• Jagged arrays: These are stored by concatenating all of the data into a single, one-dimensional array, and storing attributes to describe the shapes of each object’s data, and an offset into the beginning of each object’s data.

• Arrays with None in them: These are stored by replacing each instance of None with a magical value that shouldn’t be encountered in realistic scenarios.

Parameters

• data – An ndarray storing the data that we want to stuff into the database. These are usually dtype=Object, which is how we usually end up here in the first place.

• paramName – The parameter name that we are trying to store data for. This is mostly used for diagnostics.

See also

unpackSpecialData()

armi.bookkeeping.db.database3.unpackSpecialData(data: numpy.ndarray, attrs, paramName: str) → numpy.ndarray

Extract data from a specially-formatted HDF5 dataset into a numpy array.

This should invert the operations performed by packSpecialData().

Parameters

• data – Specially-formatted data array straight from the database.

• attrs – The attributes associated with the dataset that contained the data.

• paramName – The name of the parameter that is being unpacked. Only used for diagnostics.

Returns

An ndarray containing the closest possible representation of the data that was originally written to the database.

Return type

numpy.ndarray

See also

packSpecialData()

armi.bookkeeping.db.database3.replaceNonsenseWithNones(data: numpy.ndarray, paramName: str) → numpy.ndarray

Replace special nonsense values with None.

This essentially reverses the operations performed by replaceNonesWithNonsense().

Parameters

• data – The array from the database that contains special None nonsense values.

• paramName – The param name who’s data we are dealing with. Only used for diagnostics.

See also

replaceNonesWithNonsense()

armi.bookkeeping.db.database3.replaceNonesWithNonsense(data: numpy.ndarray, paramName: str, nones: numpy.ndarray = None) → numpy.ndarray

Replace instances of None with nonsense values that can be detected/recovered when reading.

Parameters

• data – The numpy array containing None values that need to be replaced.

• paramName – The name of the parameter who’s data we are treating. Only used for diagnostics.

• nones – An array containing the index locations on the None elements. It is a little strange to pass these, in but we find these indices to determine whether we need to call this function in the first place, so might as well pass it in, so that we don’t need to perform the operation again.

Notes

This only supports situations where the data is a straight-up None, or a valid, database-storable numpy array (or easily convertable to one (e.g. tuples/lists with numerical values)). This does not support, for instance, a numpy ndarray with some Nones in it.

For example, the following is supported:

[[1, 2, 3], None, [7, 8, 9]]

However, the following is not:

[[1, 2, 3], [4, None, 6], [7, 8, 9]]

See also

replaceNonsenseWithNones()

Reverses this operation.