reV.config.execution.HPCConfig

class HPCConfig(config_dict)[source]

Bases: BaseExecutionConfig

Class to handle HPC configuration inputs.

Parameters:: config (str | dict) – File path to config json (str), serialized json object (str), or dictionary with pre-extracted config.

Methods

`check_files`(flist)	Make sure all files in the input file list exist.
`check_overwrite_keys`(primary_key, ...)	Check for overwrite keys and raise a ConfigError if present
`clear`()
`copy`()
`fromkeys`([value])	Create a new dictionary with keys from iterable and values set to value.
`get`(key[, default])	Return the value for key if key is in the dictionary, else default.
`items`()
`keys`()
`pop`(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised
`popitem`()	Remove and return a (key, value) pair as a 2-tuple.
`resolve_path`(path)	Resolve a file path represented by the input string.
`set_self_dict`(dictlike)	Save a dict-like variable as object instance dictionary items.
`setdefault`(key[, default])	Insert key with a value of default if key is not in the dictionary.
`str_replace_and_resolve`(d, str_rep)	Perform a deep string replacement and path resolve in d.
`update`([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
`values`()

Attributes

`REQUIREMENTS`	Required keys for config
`STR_REP`	Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../
`allocation`	Get the HPC allocation property.
`conda_env`	Get conda environment to activate
`config_dir`	Get the directory that the config file is in.
`config_keys`	List of valid config keys
`feature`	Get feature request str.
`log_level`	Get user-specified "log_level" (DEBUG, INFO, WARNING, etc...).
`max_workers`	Get the max_workers property (1 runs in serial, None is all workers)
`memory_utilization_limit`	Get the node memory utilization limit property.
`module`	Get module to load if given
`name`	Get the job name, defaults to 'rev'.
`nodes`	Get the number of nodes property.
`option`	Get the hardware run option.
`sh_script`	Get the "sh_script" entry which is a string that contains extra shell script commands to run before the reV commands.
`sites_per_worker`	Get the number of sites to run per worker.

property allocation

Get the HPC allocation property.

Returns:: hpc_alloc (str) – Name of the HPC allocation account for the specified job.

property feature

Get feature request str.

Returns:: feature (str | NoneType) – Feature request string. For EAGLE, a full additional flag. Config should look like: "feature": "--depend=[state:job_id]"

property module

Get module to load if given

Returns:: module (str) – Module to load on node

property conda_env

Get conda environment to activate

Returns:: conda_env (str) – Conda environment to activate

REQUIREMENTS = (): Required keys for config

STR_REP = {'REVDIR': '/home/runner/work/reV/reV/reV', 'TESTDATADIR': '/home/runner/work/reV/reV/tests/data'}: Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../

static check_files(flist)

Make sure all files in the input file list exist.

Parameters:: flist (list) – List of files (with paths) to check existance of.

check_overwrite_keys(primary_key, *overwrite_keys)

Check for overwrite keys and raise a ConfigError if present

Parameters:

primary_key (str) – Primary key that overwrites overwrite_keys, used for error message
overwrite_keys (str) – Key(s) to overwrite

clear() → None. Remove all items from D.

property config_dir

Get the directory that the config file is in.

Returns:: config_dir (str) – Directory path that the config file is in.

property config_keys

List of valid config keys

Returns:: list

copy() → a shallow copy of D

fromkeys(value=None, /): Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /): Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

property log_level

Get user-specified “log_level” (DEBUG, INFO, WARNING, etc…).

Returns:: log_level (int) – Python logging module level (integer format) corresponding to the config-specified log level string.

property max_workers

Get the max_workers property (1 runs in serial, None is all workers)

Returns:: max_workers (int | None) – Processes per node. Default is None max_workers (all available).

property memory_utilization_limit

Get the node memory utilization limit property. Key in the config json is “memory_utilization_limit”.

Returns:: mem_util_lim (float) – Memory utilization limit (fractional). Key in the config json is “memory_utilization_limit”.

property name

Get the job name, defaults to ‘rev’.

Returns:: name (str) – reV job name.

property nodes

Get the number of nodes property.

Returns:: nodes (int) – Number of available nodes. Default is 1 node.

property option

Get the hardware run option.

Returns:: option (str) – Execution control option, e.g. local, peregrine, eagle…

pop(k[, d]) → v, remove specified key and return the corresponding value.: If key is not found, d is returned if given, otherwise KeyError is raised

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

resolve_path(path)

Resolve a file path represented by the input string.

This function resolves the input string if it resembles a path. Specifically, the string will be resolved if it starts with “./” or “..”, or it if it contains either “./” or “..” somewhere in the string body. Otherwise, the string is returned unchanged, so this function is safe to call on any string, even ones that do not resemble a path.

This method delegates the “resolving” logic to pathlib.Path.resolve(). This means the path is made absolute, symlinks are resolved, and “..” components are eliminated. If the path input starts with “./” or “..”, it is assumed to be w.r.t the config directory, not the run directory.

Parameters:: path (str) – Input file path.
Returns:: str – The resolved path.

set_self_dict(dictlike)

Save a dict-like variable as object instance dictionary items.

Parameters:: dictlike (dict) – Python namespace object to set to this dictionary-emulating class.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

property sh_script

Get the “sh_script” entry which is a string that contains extra shell script commands to run before the reV commands.

Returns:: str

property sites_per_worker

Get the number of sites to run per worker.

Returns:: sites_per_worker (int | None) – Number of sites to run per worker in a parallel scheme.

str_replace_and_resolve(d, str_rep)

Perform a deep string replacement and path resolve in d.

Parameters:

d (dict) – Config dictionary potentially containing strings to replace and/or paths to resolve.
str_rep (dict) – Replacement mapping where keys are strings to search for and values are the new values.

Returns:

d (dict) – Config dictionary with updated strings.

update([E, ]**F) → None. Update D from dict/iterable E and F.: If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values