JADE Configuration

This page describes the options available in a JADE configuration for generic CLI commands (config.json). Most options also apply to other extensions.

Main Level

  • user_data: Any global data that you would like to be available to jobs at runtime. Must be serializable in JSON format.

  • submission_groups: Optional array of SubmissionGroup objects. Allows submitting jobs with differing batch sizes and HPC parameters. If set, each job must be assigned to a group. Refer to Submission Group Behaviors for additional information.

  • jobs: Array of job definitions. See below.

Per-Job Definition

Each object in the array of jobs at the main level must be an instance of GenericCommandParameters.

GenericCommandParameters

Property

Type

Description

Required

Default

name

string

If not set Jade will use the job_id converted to a string. Must be unique.

False

use_multi_node_manager

boolean

If true JADE will run this job with its multi-node manager.

False

False

spark_config

Any

If enabled JADE will run this job on a Spark cluster.

False

command

string

Command that can be invoked in a terminal (shell characters not allowed)

True

blocked_by

array

Array of job names that must complete before this one can start.

False

[]

cancel_on_blocking_job_failure

boolean

If true JADE will cancel this job if any of its blocking jobs fail.

False

False

estimated_run_minutes

integer

JADE will use this value along with num-parallel-processes-per-node and walltime to build per-node batches of jobs if time-based-batching is enabled.

False

submission_group

string

Optional name of a submission group

False

default

append_job_name

boolean

If true JADE will append –jade-job-name=X where X is the job’s name.

False

False

append_output_dir

boolean

If true JADE will append –jade-runtime-output=X where X is the output directory specified in jade submit-jobs.

False

False

ext

object

User-defined extension data to be used at runtime. Must be serializable in JSON format.

False

{}

job_id

integer

Unique job identifier, generated by JADE

False

extension

string

job extension type, generated by JADE

False

generic_command

Submission Group Behaviors

JADE implements these behaviors when submission groups are defined.

JADE will raise an exception if the following parameters are not the same across all submission groups:

  • max_nodes

  • poll_interval

JADE will override the following parameters in the submission groups with parameters specified on the command line for jade submit-jobs:

  • dry_run

  • generate_reports

  • resource_monitor_interval

  • verbose

The following parameters are completely controlled by the group:

  • hpc_config

  • num_processes

  • per_node_batch_size

  • time_based_batching

  • try_add_blocked_jobs