reV.config.execution.SlurmConfig

class SlurmConfig(config_dict)

Bases: HPCConfig

Class to handle SLURM (Eagle) configuration inputs.

Parameters:

config (str | dict) – File path to config json (str), serialized json object (str), or dictionary with pre-extracted config.

Methods

check_files(flist)	Make sure all files in the input file list exist.
check_overwrite_keys(primary_key, ...)	Check for overwrite keys and raise a ConfigError if present
clear()
copy()
fromkeys([value])	Create a new dictionary with keys from iterable and values set to value.
get(key[, default])	Return the value for key if key is in the dictionary, else default.
items()
keys()
pop(k[,d])	If key is not found, d is returned if given, otherwise KeyError is raised
popitem()	Remove and return a (key, value) pair as a 2-tuple.
resolve_path(path)	Resolve a file path represented by the input string.
set_self_dict(dictlike)	Save a dict-like variable as object instance dictionary items.
setdefault(key[, default])	Insert key with a value of default if key is not in the dictionary.
str_replace_and_resolve(d, str_rep)	Perform a deep string replacement and path resolve in d.
update([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values()

Attributes

REQUIREMENTS	Required keys for config
STR_REP	Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../
allocation	Get the HPC allocation property.
conda_env	Get conda environment to activate
config_dir	Get the directory that the config file is in.
config_keys	List of valid config keys
feature	Get feature request str.
log_level	Get user-specified "log_level" (DEBUG, INFO, WARNING, etc...).
max_workers	Get the max_workers property (1 runs in serial, None is all workers)
memory	Get the requested Eagle node "memory" value in GB or can be None.
memory_utilization_limit	Get the node memory utilization limit property.
module	Get module to load if given
name	Get the job name, defaults to 'rev'.
nodes	Get the number of nodes property.
option	Get the hardware run option.
sh_script	Get the "sh_script" entry which is a string that contains extra shell script commands to run before the reV commands.
sites_per_worker	Get the number of sites to run per worker.
walltime	Get the requested Eagle node "walltime" value.

property memory

Get the requested Eagle node “memory” value in GB or can be None.

Returns:

_hpc_node_mem (int | None) – Requested node memory in GB.

property walltime

Get the requested Eagle node “walltime” value.

Returns:

_hpc_walltime (int) – Requested single node job time in hours.

REQUIREMENTS = ()

Required keys for config

STR_REP = {'REVDIR': '/home/runner/work/reV/reV/reV', 'TESTDATADIR': '/home/runner/work/reV/reV/tests/data'}

Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../

property allocation

Get the HPC allocation property.

Returns:

hpc_alloc (str) – Name of the HPC allocation account for the specified job.

static check_files(flist)

Make sure all files in the input file list exist.

Parameters:

flist (list) – List of files (with paths) to check existance of.

check_overwrite_keys(primary_key, *overwrite_keys)

Check for overwrite keys and raise a ConfigError if present

Parameters:

• primary_key (str) – Primary key that overwrites overwrite_keys, used for error message

• overwrite_keys (str) – Key(s) to overwrite

clear() → None.  Remove all items from D.

property conda_env

Get conda environment to activate

Returns:

conda_env (str) – Conda environment to activate

property config_dir

Get the directory that the config file is in.

Returns:

config_dir (str) – Directory path that the config file is in.

property config_keys

List of valid config keys

Returns:

list

copy() → a shallow copy of D

property feature

Get feature request str.

Returns:

feature (str | NoneType) – Feature request string. For EAGLE, a full additional flag. Config should look like: "feature": "--depend=[state:job_id]"

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

property log_level

Get user-specified “log_level” (DEBUG, INFO, WARNING, etc…).

Returns:

log_level (int) – Python logging module level (integer format) corresponding to the config-specified log level string.

property max_workers

Get the max_workers property (1 runs in serial, None is all workers)

Returns:

max_workers (int | None) – Processes per node. Default is None max_workers (all available).

property memory_utilization_limit

Get the node memory utilization limit property. Key in the config json is “memory_utilization_limit”.

Returns:

mem_util_lim (float) – Memory utilization limit (fractional). Key in the config json is “memory_utilization_limit”.

property module

Get module to load if given

Returns:

module (str) – Module to load on node

property name

Get the job name, defaults to ‘rev’.

Returns:

name (str) – reV job name.

property nodes

Get the number of nodes property.

Returns:

nodes (int) – Number of available nodes. Default is 1 node.

property option

Get the hardware run option.

Returns:

option (str) – Execution control option, e.g. local, peregrine, eagle…

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

resolve_path(path)

Resolve a file path represented by the input string.

This function resolves the input string if it resembles a path. Specifically, the string will be resolved if it starts with “./” or “..”, or it if it contains either “./” or “..” somewhere in the string body. Otherwise, the string is returned unchanged, so this function is safe to call on any string, even ones that do not resemble a path.

This method delegates the “resolving” logic to pathlib.Path.resolve(). This means the path is made absolute, symlinks are resolved, and “..” components are eliminated. If the path input starts with “./” or “..”, it is assumed to be w.r.t the config directory, not the run directory.

Parameters:

path (str) – Input file path.

Returns:

str – The resolved path.

set_self_dict(dictlike)

Save a dict-like variable as object instance dictionary items.

Parameters:

dictlike (dict) – Python namespace object to set to this dictionary-emulating class.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

property sh_script

Get the “sh_script” entry which is a string that contains extra shell script commands to run before the reV commands.

Returns:

str

property sites_per_worker

Get the number of sites to run per worker.

Returns:

sites_per_worker (int | None) – Number of sites to run per worker in a parallel scheme.

str_replace_and_resolve(d, str_rep)

Perform a deep string replacement and path resolve in d.

Parameters:

• d (dict) – Config dictionary potentially containing strings to replace and/or paths to resolve.

• str_rep (dict) – Replacement mapping where keys are strings to search for and values are the new values.

Returns:

d (dict) – Config dictionary with updated strings.

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values