reV multi-year

Execute the multi-year step from a config file.

reV multi-year combines reV generation data from multiple years (typically stored in separate files) into a single multi-year file. Each dataset in the multi-year file is labeled with the corresponding years, and multi-year averages of the yearly datasets are also computed.

The general structure for calling this CLI command is given below (add --help to print help info to the terminal).

reV multi-year [OPTIONS]

Options

-c, --config_file <config_file>

Required Path to the multi-year configuration file. Below is a sample template config

{
    "execution_control": {
        "option": "local",
        "allocation": "[REQUIRED IF ON HPC]",
        "walltime": "[REQUIRED IF ON HPC]",
        "qos": "normal",
        "memory": null,
        "queue": null,
        "feature": null,
        "conda_env": null,
        "module": null,
        "sh_script": null,
        "num_test_nodes": null
    },
    "log_directory": "./logs",
    "log_level": "INFO",
    "groups": "[REQUIRED]",
    "clobber": true
}

Parameters

execution_controldict

Dictionary containing execution control arguments. Allowed arguments are:

option:

({‘local’, ‘kestrel’, ‘eagle’, ‘awspc’, ‘slurm’, ‘peregrine’}) Hardware run option. Determines the type of job scheduler to use as well as the base AU cost. The “slurm” option is a catchall for HPC systems that use the SLURM scheduler and should only be used if desired hardware is not listed above. If “local”, no other HPC-specific keys in are required in execution_control (they are ignored if provided).

allocation:

(str) HPC project (allocation) handle.

walltime:

(int) Node walltime request in hours.

qos:

(str, optional) Quality-of-service specifier. For Kestrel users: This should be one of {‘standby’, ‘normal’, ‘high’}. Note that ‘high’ priority doubles the AU cost. By default, "normal".

memory:

(int, optional) Node memory max limit (in GB). By default, None, which uses the scheduler’s default memory limit. For Kestrel users: If you would like to use the full node memory, leave this argument unspecified (or set to None) if you are running on standard nodes. However, if you would like to use the bigmem nodes, you must specify the full upper limit of memory you would like for your job, otherwise you will be limited to the standard node memory size (250GB).

queue:

(str, optional; PBS ONLY) HPC queue to submit job to. Examples include: ‘debug’, ‘short’, ‘batch’, ‘batch-h’, ‘long’, etc. By default, None, which uses “test_queue”.

feature:

(str, optional) Additional flags for SLURM job (e.g. “-p debug”). By default, None, which does not specify any additional flags.

conda_env:

(str, optional) Name of conda environment to activate. By default, None, which does not load any environments.

module:

(str, optional) Module to load. By default, None, which does not load any modules.

sh_script:

(str, optional) Extra shell script to run before command call. By default, None, which does not run any scripts.

num_test_nodes:

(str, optional) Number of nodes to submit before terminating the submission process. This can be used to test a new submission configuration without sumbitting all nodes (i.e. only running a handful to ensure the inputs are specified correctly and the outputs look reasonable). By default, None, which submits all node jobs.

Only the option key is required for local execution. For execution on the HPC, the allocation and walltime keys are also required. All other options are populated with default values, as seen above.

log_directorystr

Path to directory where logs should be written. Path can be relative and does not have to exist on disk (it will be created if missing). By default, "./logs".

log_level{“DEBUG”, “INFO”, “WARNING”, “ERROR”}

String representation of desired logger verbosity. Suitable options are DEBUG (most verbose), INFO (moderately verbose), WARNING (only log warnings and errors), and ERROR (only log errors). By default, "INFO".

groupsdict

Dictionary of collection groups and their parameters. This should be a dictionary mapping group names (keys) to a set of key word arguments (values) that can be used to initialize MultiYearGroup (excluding the required name and out_dir inputs, which are populated automatically). For example:

groups = {
    "none": {
        "dsets": [
            "cf_profile",
            "cf_mean",
            "ghi_mean",
            "lcoe_fcr",
        ],
        "source_dir": "./",
        "source_prefix": "",
        "pass_through_dsets": [
            "capital_cost",
            "fixed_operating_cost",
            "system_capacity",
            "fixed_charge_rate",
            "variable_operating_cost",
        ]
    },
    "solar_group": {
        "source_files": "PIPELINE",
        "dsets": [
            "cf_profile_ac",
            "cf_mean_ac",
            "ac",
            "dc",
            "clipped_power"
        ],
        "pass_through_dsets": [
            "system_capacity_ac",
            "dc_ac_ratio"
        ]
    },
    ...
}

The group names will be used as the HDF5 file group name under which the collected data will be stored. You can have exactly one group with the name "none" for a “no group” collection (this is typically what you want and all you need to specify).

clobberbool, optional

Flag to purge the multi-year output file prior to running the multi-year collection step if the file already exists on disk. This ensures the data is always freshly collected from the single-year files. If False, then datasets in the existing file will not be overwritten with (potentially new/updated) data from the single-year files. By default, True.

Note that you may remove any keys with a null value if you do not intend to update them yourself.