HPC Profiles Reference

Complete reference for HPC profile system and CLI commands.

Overview

HPC profiles contain pre-configured knowledge about High-Performance Computing systems, enabling automatic Slurm scheduler generation based on job resource requirements.

CLI Commands

torc hpc list

List all available HPC profiles.

torc hpc list [OPTIONS]

Options:

Output columns:

• Name: Profile identifier used in commands
• Display Name: Human-readable name
• Partitions: Number of configured partitions
• Detected: Whether current system matches this profile

torc hpc detect

Detect the current HPC system.

torc hpc detect [OPTIONS]

Options:

Returns the detected profile name, or indicates no match.

torc hpc show

Display detailed information about an HPC profile.

torc hpc show <PROFILE> [OPTIONS]

Arguments:

Options:

torc hpc partitions

List partitions for an HPC profile.

torc hpc partitions <PROFILE> [OPTIONS]

Arguments:

Options:

Output columns:

• Name: Partition name
• CPUs/Node: CPU cores per node
• Mem/Node: Memory per node
• Max Walltime: Maximum job duration
• GPUs: GPU count and type (if applicable)
• Shared: Whether partition supports shared jobs
• Notes: Special requirements or features

torc hpc match

Find partitions matching resource requirements.

torc hpc match <PROFILE> [OPTIONS]

Arguments:

Options:

Memory format: <number><unit> where unit is k, m, g, or t (case-insensitive).

Walltime formats:

• HH:MM:SS (e.g., 04:00:00)
• <N>h (e.g., 4h)
• <N>m (e.g., 30m)
• <N>s (e.g., 3600s)

torc slurm generate

Generate Slurm schedulers for a workflow based on job resource requirements.

torc slurm generate [OPTIONS] --account <ACCOUNT> <WORKFLOW_FILE>

Arguments:

Options:

Generated artifacts:

1. Slurm schedulers: One for each unique resource requirement
2. Job scheduler assignments: Each job linked to appropriate scheduler
3. Workflow actions: on_workflow_start/schedule_nodes actions (unless --no-actions)

Scheduler naming: <resource_requirement_name>_scheduler

Built-in Profiles

NREL Kestrel

Profile name: kestrel

Detection: Environment variable NREL_CLUSTER=kestrel

Partitions:

Node specifications:

• Standard nodes: 104 cores (2x Intel Xeon Sapphire Rapids), 240 GB RAM
• GPU nodes: 4x NVIDIA H100 80GB HBM3, 128 cores, 2 TB RAM

Configuration

Custom Profiles

Don’t see your HPC? Please request built-in support so everyone benefits. See the Custom HPC Profile Tutorial for creating a profile while you wait.

Define custom profiles in your Torc configuration file:

# ~/.config/torc/config.toml

[client.hpc.custom_profiles.mycluster]
display_name = "My Cluster"
description = "Description of the cluster"
detect_env_var = "CLUSTER_NAME=mycluster"
detect_hostname = ".*\\.mycluster\\.org"
default_account = "myproject"

[[client.hpc.custom_profiles.mycluster.partitions]]
name = "compute"
cpus_per_node = 64
memory_mb = 256000
max_walltime_secs = 172800
shared = false

[[client.hpc.custom_profiles.mycluster.partitions]]
name = "gpu"
cpus_per_node = 32
memory_mb = 128000
max_walltime_secs = 86400
gpus_per_node = 4
gpu_type = "A100"
shared = false

Profile Override

Override settings for built-in profiles:

[client.hpc.profile_overrides.kestrel]
default_account = "my_default_account"

Configuration Options

[client.hpc] Section:

Profile override options:

Custom profile options:

Partition options:

Resource Matching Algorithm

When generating schedulers, Torc uses this algorithm to match resource requirements to partitions:

1. Filter by resources: Partitions must satisfy:

   • CPUs >= required CPUs
   • Memory >= required memory
   • GPUs >= required GPUs (if specified)
   • Max walltime >= required runtime

2. Exclude debug partitions: Unless no other partition matches

3. Prefer best fit:

   • Partitions that exactly match resource needs
   • Non-shared partitions over shared
   • Shorter max walltime over longer

4. Handle special requirements:

   • GPU jobs only match GPU partitions
   • Respect requires_explicit_request flag
   • Honor min_nodes constraints

Generated Scheduler Format

Example generated Slurm scheduler:

slurm_schedulers:
  - name: medium_scheduler
    account: myproject
    nodes: 1
    mem: 64g
    walltime: 04:00:00
    gres: null
    partition: null  # Let Slurm choose based on resources

Corresponding workflow action:

actions:
  - trigger_type: on_workflow_start
    action_type: schedule_nodes
    scheduler: medium_scheduler
    scheduler_type: slurm
    num_allocations: 1

Runtime Format Parsing

Resource requirements use ISO 8601 duration format for runtime:

Generated walltime uses HH:MM:SS format (e.g., 04:00:00).

See Also

• Working with HPC Profiles
• Custom HPC Profile Tutorial
• Working with Slurm
• Resource Requirements
• Configuration Reference