nsrdb.utilities.cli.BaseCLI

class BaseCLI[source]

Bases: object

Base CLI class used to create CLI for modules in ModuleName

Methods

check_module_name(module_name)

Make sure module_name is a valid member of the ModuleName class

from_config_preflight(ctx, module_name, ...)

Parse conifg file prior to running nsrdb module.

get_cmd_args(module_name, config)

Get module specific kwargs.

get_status_cmd(config, pipeline_step)

Append status file command to command for executing given module

kickoff_job(ctx, module_name, func, config)

Run nsrdb module either locally or on HPC.

kickoff_local_job(ctx, module_name, cmd)

Run nsrdb module locally.

kickoff_multichunk(ctx, module_name, func, ...)

Kick off jobs for multiple chunks.

kickoff_multiday(ctx, module_name, func, ...)

Kick off jobs for multiple days.

kickoff_single(ctx, module_name, func, ...)

Kick off single job.

kickoff_slurm_job(ctx, module_name, cmd[, ...])

Run nsrdb module on HPC via SLURM job submission.

running_or_done(status_dir, pipeline_step, ...)

Check if job is running or complete.
classmethod from_config_preflight(ctx, module_name, config, verbose, pipeline_step=None)[source]

Parse conifg file prior to running nsrdb module.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • config (str) – Path to config file or dictionary providing all needed inputs to module_class

  • verbose (bool) – Whether to run in verbose mode.

  • pipeline_step (str, optional) – Name of the pipeline step being run. If None, the pipeline_step will be set to the module_name, mimicking old reV behavior. By default, None.

Returns:

config (dict) – Dictionary with module specifc inputs only.

static get_cmd_args(module_name, config)[source]

Get module specific kwargs.

classmethod check_module_name(module_name)[source]

Make sure module_name is a valid member of the ModuleName class

static running_or_done(status_dir, pipeline_step, job_name, subprocess_manager=None)[source]

Check if job is running or complete.

classmethod kickoff_slurm_job(ctx, module_name, cmd, option='kestrel', alloc='nsrdb', memory=None, walltime=4, feature=None, stdout_path='./stdout/')[source]

Run nsrdb module on HPC via SLURM job submission.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • cmd (str) –

    Command to be submitted in SLURM shell script. Example:

    ‘python -m nsrdb.cli <module_name> -c <config>’

  • option (str) – Hardware option. e.g. “kestrel” or “slurm”.

  • alloc (str) – HPC project (allocation) handle. Example: ‘nsrdb’.

  • memory (int) – Node memory request in GB.

  • walltime (float) – Node walltime request in hours.

  • feature (str) – Additional flags for SLURM job. Format is “–qos=high” or “–depend=[state:job_id]”. Default is None.

  • stdout_path (str) – Path to print .stdout and .stderr files.

  • pipeline_step (str, optional) – Name of the pipeline step being run. If None, the pipeline_step will be set to the module_name, mimicking old reV behavior. By default, None.

classmethod kickoff_local_job(ctx, module_name, cmd)[source]

Run nsrdb module locally.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • cmd (str) –

    Command to be submitted in shell script. Example:

    ‘python -m nsrdb.cli <module_name> -c <config>’

classmethod get_status_cmd(config, pipeline_step)[source]

Append status file command to command for executing given module

Parameters:

  • config (dict) – nsrdb config with all necessary args and kwargs to run given module.

  • pipeline_step (str) – Name of the pipeline step being run.

  • cmd (str) – String including command to execute given module.

Returns:

cmd (str) – Command string with status file command included if job_name is not None

classmethod kickoff_job(ctx, module_name, func, config, log_id=None)[source]

Run nsrdb module either locally or on HPC.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • func (Callable) – Function used to run module. e.g. NSRDB.run_data_model()

  • config (dict) – nsrdb config with all necessary args and kwargs to run given module.

  • log_id (str | None) – String id to append to base log file if this job is part of a multi job kickoff. None is used is this is just a single job.

classmethod kickoff_single(ctx, module_name, func, config, verbose, pipeline_step=None)[source]

Kick off single job.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • func (Callable) – Function used to run module. e.g. NSRDB.run_data_model()

  • config (str | dict) – Path to nsrdb config file or a dictionary with all necessary args and kwargs to run given module

  • verbose (bool) – Flag to turn on debug logging

  • pipeline_step (str, optional) – Name of the pipeline step being run. If None, the pipeline_step will be set to the module_name, mimicking old reV behavior. By default, None.

classmethod kickoff_multiday(ctx, module_name, func, config, verbose, pipeline_step=None)[source]

Kick off jobs for multiple days.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • func (Callable) – Function used to run module. e.g. NSRDB.run_data_model()

  • config (str | dict) – Path to nsrdb config file or a dictionary with all necessary args and kwargs to run given module

  • verbose (bool) – Flag to turn on debug logging

  • pipeline_step (str, optional) – Name of the pipeline step being run. If None, the pipeline_step will be set to the module_name, mimicking old reV behavior. By default, None.

classmethod kickoff_multichunk(ctx, module_name, func, config, verbose, pipeline_step=None)[source]

Kick off jobs for multiple chunks.

Parameters:

  • ctx (click.pass_context) – Click context object where ctx.obj is a dictionary

  • module_name (str) – Module name string from nsrdb.utilities.ModuleName.

  • func (Callable) – Function used to run module. e.g. NSRDB.gap_fill_clouds()

  • config (str | dict) – Path to nsrdb config file or a dictionary with all necessary args and kwargs to run given module

  • verbose (bool) – Flag to turn on debug logging

  • pipeline_step (str, optional) – Name of the pipeline step being run. If None, the pipeline_step will be set to the module_name, mimicking old reV behavior. By default, None.