reV.config.execution.SlurmConfig

class SlurmConfig(config_dict)[source]

Bases: HPCConfig

Class to handle SLURM (Eagle) configuration inputs.

Parameters:

config (str | dict) – File path to config json (str), serialized json object (str), or dictionary with pre-extracted config.

Methods

check_files(flist)

Make sure all files in the input file list exist.

check_overwrite_keys(primary_key, ...)

Check for overwrite keys and raise a ConfigError if present

clear()

copy()

fromkeys([value])

Create a new dictionary with keys from iterable and values set to value.

get(key[, default])

Return the value for key if key is in the dictionary, else default.

items()

keys()

pop(k[,d])

If key is not found, d is returned if given, otherwise KeyError is raised

popitem()

Remove and return a (key, value) pair as a 2-tuple.

resolve_path(path)

Resolve a file path represented by the input string.

set_self_dict(dictlike)

Save a dict-like variable as object instance dictionary items.

setdefault(key[, default])

Insert key with a value of default if key is not in the dictionary.

str_replace_and_resolve(d, str_rep)

Perform a deep string replacement and path resolve in d.

update([E, ]**F)

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

Attributes

REQUIREMENTS

Required keys for config

STR_REP

Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../

allocation

Get the HPC allocation property.

conda_env

Get conda environment to activate

config_dir

Get the directory that the config file is in.

config_keys

List of valid config keys

feature

Get feature request str.

log_level

Get user-specified "log_level" (DEBUG, INFO, WARNING, etc...).

max_workers

Get the max_workers property (1 runs in serial, None is all workers)

memory

Get the requested Eagle node "memory" value in GB or can be None.

memory_utilization_limit

Get the node memory utilization limit property.

module

Get module to load if given

name

Get the job name, defaults to 'rev'.

nodes

Get the number of nodes property.

option

Get the hardware run option.

sh_script

Get the "sh_script" entry which is a string that contains extra shell script commands to run before the reV commands.

sites_per_worker

Get the number of sites to run per worker.

walltime

Get the requested Eagle node "walltime" value.

property memory

Get the requested Eagle node “memory” value in GB or can be None.

Returns:

_hpc_node_mem (int | None) – Requested node memory in GB.

property walltime

Get the requested Eagle node “walltime” value.

Returns:

_hpc_walltime (int) – Requested single node job time in hours.

REQUIREMENTS = ()

Required keys for config

STR_REP = {'REVDIR': '/home/runner/work/reV/reV/reV', 'TESTDATADIR': '/home/runner/work/reV/reV/tests/data'}

Mapping of config inputs (keys) to desired replacements (values) in addition to relative file paths as demarcated by ./ and ../

property allocation

Get the HPC allocation property.

Returns:

hpc_alloc (str) – Name of the HPC allocation account for the specified job.

static check_files(flist)

Make sure all files in the input file list exist.

Parameters:

flist (list) – List of files (with paths) to check existance of.

check_overwrite_keys(primary_key, *overwrite_keys)

Check for overwrite keys and raise a ConfigError if present

Parameters:
  • primary_key (str) – Primary key that overwrites overwrite_keys, used for error message

  • overwrite_keys (str) – Key(s) to overwrite

clear() None.  Remove all items from D.
property conda_env

Get conda environment to activate

Returns:

conda_env (str) – Conda environment to activate

property config_dir

Get the directory that the config file is in.

Returns:

config_dir (str) – Directory path that the config file is in.

property config_keys

List of valid config keys

Returns:

list

copy() a shallow copy of D
property feature

Get feature request str.

Returns:

feature (str | NoneType) – Feature request string. For EAGLE, a full additional flag. Config should look like: "feature": "--depend=[state:job_id]"

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items() a set-like object providing a view on D's items
keys() a set-like object providing a view on D's keys
property log_level

Get user-specified “log_level” (DEBUG, INFO, WARNING, etc…).

Returns:

log_level (int) – Python logging module level (integer format) corresponding to the config-specified log level string.

property max_workers

Get the max_workers property (1 runs in serial, None is all workers)

Returns:

max_workers (int | None) – Processes per node. Default is None max_workers (all available).

property memory_utilization_limit

Get the node memory utilization limit property. Key in the config json is “memory_utilization_limit”.

Returns:

mem_util_lim (float) – Memory utilization limit (fractional). Key in the config json is “memory_utilization_limit”.

property module

Get module to load if given

Returns:

module (str) – Module to load on node

property name

Get the job name, defaults to ‘rev’.

Returns:

name (str) – reV job name.

property nodes

Get the number of nodes property.

Returns:

nodes (int) – Number of available nodes. Default is 1 node.

property option

Get the hardware run option.

Returns:

option (str) – Execution control option, e.g. local, peregrine, eagle…

pop(k[, d]) v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

resolve_path(path)

Resolve a file path represented by the input string.

This function resolves the input string if it resembles a path. Specifically, the string will be resolved if it starts with “./” or “..”, or it if it contains either “./” or “..” somewhere in the string body. Otherwise, the string is returned unchanged, so this function is safe to call on any string, even ones that do not resemble a path.

This method delegates the “resolving” logic to pathlib.Path.resolve(). This means the path is made absolute, symlinks are resolved, and “..” components are eliminated. If the path input starts with “./” or “..”, it is assumed to be w.r.t the config directory, not the run directory.

Parameters:

path (str) – Input file path.

Returns:

str – The resolved path.

set_self_dict(dictlike)

Save a dict-like variable as object instance dictionary items.

Parameters:

dictlike (dict) – Python namespace object to set to this dictionary-emulating class.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

property sh_script

Get the “sh_script” entry which is a string that contains extra shell script commands to run before the reV commands.

Returns:

str

property sites_per_worker

Get the number of sites to run per worker.

Returns:

sites_per_worker (int | None) – Number of sites to run per worker in a parallel scheme.

str_replace_and_resolve(d, str_rep)

Perform a deep string replacement and path resolve in d.

Parameters:
  • d (dict) – Config dictionary potentially containing strings to replace and/or paths to resolve.

  • str_rep (dict) – Replacement mapping where keys are strings to search for and values are the new values.

Returns:

d (dict) – Config dictionary with updated strings.

update([E, ]**F) None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() an object providing a view on D's values