Skip to content

Running Models

Please follow the setup instructions before running the examples. Example scripts for new users are available within the master branch.

Note

Sometimes newer modules may be availabe in a test directory which is hidden by default from the general user base. This obscured release is done to iron out any bugs that may arise during the installation and use of the module while avoiding breaking users existing jobs and workflows. You can use these test modules by running

module use /nopt/nrel/apps/modules/test/modulefiles
module avail

This should display all of the test modules available in addition to the defaults. We encourage you to reach out to us at HPC-Help@nrel.gov for access if you would like access to these modules.

Example Run#

We will load the requisite modules for running PLEXOS 9.2R06 for this example. Please see the module compatibility chart for loading the correct modules

module load gurobi/10.0.2
module load plexos/9.200R06

Recall that we can only use the Gurobi solver while running the PLEXOS on the NREL cluster. Now that we have the modules loaded, PLEXOS can be called as follows

$PLEXOS/PLEXOS64 -n 5_bus_system_v2.xml -m 2024_yr_15percPV_MT_Gurobi

The command above assumes that we are running the model 2024_yr_15percPV_MT_Gurobi from file 5_bus_system_v2.xml. PLEXOS 9.0RX requires validating user-credentials for a local PLEXOS account for each run. Therefore, if we ran the above command in an interactive session, we would need to enter the following username and password

username : nrelplexos
password : Nr3lplex0s

Fortunately, we can bypass the prompt for a local PLEXOS account username and password (useful for slurm batch jobs) by passing them as command line arguments as follows.

$PLEXOS/PLEXOS64 -n 5_bus_system_v2.xml -m 2024_yr_15percPV_MT_Gurobi -cu nrelplexos -cp Nr3lplex0s

Warning

Not providing the username and password in batch jobs WILL cause your jobs to fail.

Example Scripts#

The example scripts are available here. Please clone the repository to run those examples.

Note

The slurm output files generated by PLEXOS may not load correctly because of special characters that PLEXOS output introduces. To remove thoses special characters, open the slurm output file and run the following command

# On PC
:%s/<CTRL-2>//g
# On Mac
%s/<CTRL-SHIFT-2>//g

<CTRL-2> or <CTRL-SHIFT-2> should generate the symbol ^@ that is messing up the output. Please refer to this stack exchange post for further information

1: Basic Functionality Test#

The basic functionality test is the same as the example run in the section above. We will

  1. Request an interactive node
  2. Go to the correct example directory
  3. Run the PLEXOS example interactively
Simple 5 bus problem 
# Request an interactive session on the cluster
salloc -N 1 --account=<your_hpc_allocation_name> --time=1:00:00 --partition=debug

# Go to the working directory that contains the 5_bus_system_v2.xml example
cd /to/you/XML/file/

# Load the requisite modules
module load gurobi/10.0.2
module load plexos/9.200R06

# Finally run the PLEXOS executable
$PLEXOS/PLEXOS64 -n 5_bus_system_v2.xml -m 2024_yr_15percPV_MT_Gurobi -cu nrelplexos -cp Nr3lplex0s

2: Simple batch script submission#

We will run the same example by submitting the job to the SLURM queue. This example uses the batch file submit_simple.sh. In order to run this example as is, run the following commands

Submit job in a batch file. 
# SSH into Kestrel or your cluster of choice
ssh $USER@kestrel.hpc.nrel.gov

# Clone the HPC master branch in your scratch folder
cd /scratch/${USER}/
git clone git@github.com:NREL/HPC.git

# Go to the appropriate folder and submit the job on the HPC
cd HPC/applications/plexos/RunFiles
sbatch -A account_name --mail-user=your.email@nrel.gov submit_simple.sh

3: Enhanced batch script submission#

This builds upon the previous example where it tries to run the same model as before, but adds redundancy where the job doesn't fail if a license is not found. The submission script submit_enhanced.sh attempts to re-run the job after waiting 120 seconds for each attempt.

Slightly enhanced batch submission script 
# Skip this if you already have the repo cloned in your scratch directory
ssh $USER@kestrel.hpc.nrel.gov
cd /scratch/${USER}/
git clone git@github.com:NREL/HPC.git

# Go into the appropriate directory
cd /scratch/${USER}/HPC/applications/plexos/RunFiles
sbatch -A account_name --mail-user=your.email@nrel.gov submit_enhanced.sh

4: Submitting multiple PLEXOS jobs#

This example demonstrates how to submit multiple PLEXOS jobs. The model names are present in a file called models.txt. submit_multiple.sh is simply a wrapper that calls the batch file submit_plexos.sh.

Submit multiple PLEXOS jobs 
# Skip this if you already have the repo cloned in your scratch directory
ssh $USER@kestrel.hpc.nrel.gov
cd /scratch/${USER}/
git clone git@github.com:NREL/HPC.git

# Go into the appropriate directory
cd /scratch/${USER}/HPC/applications/plexos/RunFiles
./submit_multiple.sh 5_bus_system_v2.xml models.txt

5: Running PLEXOS with SLURM array jobs#

This example demonstrates the use of SLURM job arrays to run multiple PLEXOS jobs using the script submit_job_array.sh

Submit Slurm job-array for PLEXOS 
# Skip this if you already have the repo cloned in your scratch directory
ssh $USER@kestrel.hpc.nrel.gov
cd /scratch/${USER}/
git clone git@github.com:NREL/HPC.git

# Go into the appropriate directory
cd /scratch/${USER}/HPC/applications/plexos/RunFiles
export filename=5_bus_system_v2 # Export the XML dataset name
export models.txt # Export the file that contains the models names within the XML dataset
sbatch -A account_name -t 5 --mail-user=your.email@nrel.gov --array=1-4 submit_job_array.sh