jade.jobs.pipeline_manager.PipelineManager

class jade.jobs.pipeline_manager.PipelineManager(config_file, output)

Bases: object

Manages the execution of a pipeline of JADE configurations.

Methods

create(config_file, output)	Create a new PipelineManager.
create_config_from_commands(...)	Create a new PipelineConfig from a list of jade config commands.
create_config_from_files(config_files, ...)	Create a new PipelineConfig from a list of config files.
get_stage_config_file_name(stage_num)	Return the filename of a stage config file.
get_stage_config_file_path(output, stage_num)	Return the path to a stage config file.
get_stage_output_name(stage_num)	Return the output directory name of a stage.
get_stage_output_path(output, stage_num)	Return the path to the output directory of a stage.
load(output)	Load a PipelineManager from an execution directory.
submit_next_stage(stage_num[, return_code])	Submit the next stage of the pipeline for execution.

Attributes

CONFIG_FILENAME
config	Return the pipeline config.
path	Return the pipeline directory.
stage_num	Return the current stage index
stages	Return the stages in the pipeline.

classmethod create(config_file, output)

Create a new PipelineManager.

Parameters:

• config_file (str)

• output (str) – output directory for execution

Return type:

PipelineManager

classmethod load(output)

Load a PipelineManager from an execution directory.

Parameters:

output (str)

Return type:

PipelineManager

submit_next_stage(stage_num, return_code=None)

Submit the next stage of the pipeline for execution.

Parameters:

• stage_num (int) – stage number to submit

• return_code (int) – status of the previous stage if this wasn’t the first

static create_config_from_files(config_files, pipeline_config_file, submit_params)

Create a new PipelineConfig from a list of config files.

Parameters:

• config_files (list) – Jade config files for each stage of the pipeline

• pipeline_config_file (str)

• submit_params (SubmitterParams)

Return type:

PipelineConfig

static create_config_from_commands(auto_config_cmds, pipeline_config_file, submit_params)

Create a new PipelineConfig from a list of jade config commands.

Parameters:

• auto_config_cmds (list) – list of commands (str) used to create Jade configs.

• pipeline_config_file (str)

• submit_params (SubmitterParams)

Return type:

PipelineConfig

property config

Return the pipeline config.

Return type:

PipelineConfig

property path

Return the pipeline directory.

Return type:

int

property stage_num

Return the current stage index

Return type:

int

property stages

Return the stages in the pipeline.

Returns:

list of PipelineStage

Return type:

list

static get_stage_config_file_name(stage_num)

Return the filename of a stage config file.

static get_stage_config_file_path(output, stage_num)

Return the path to a stage config file.

static get_stage_output_name(stage_num)

Return the output directory name of a stage.

static get_stage_output_path(output, stage_num)

Return the path to the output directory of a stage.