Skip to content

How to Use the WRF Application Software#

Documentation: Weather Research Framework (WRF) Model

The WRF model is a state of the art mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications.

Getting Started#

This section provides the minimum amount of information necessary to successfully run a WRF job on the NREL Kestrel cluster. First, we show how to use WRF given that we may have different versions of WRF in different toolchains already built and available as modules.

% module avail wrf
     wrf/4.2.2-cray (D)    
     wrf/4.2.2-intel

The module avail wrf command shows which WRF module(s) are available for different versions of WRF built with different toolchains, at a given time. At this time, the version 4.2.2 is built with the Cray and Intel toolchains as currently available. Users are then free to choose any of the module versions available for use. Currently there are no modules to run WRF on GPUs, but there is current effort underway to make that available on future systems.

Next, we look at how to use the WRF module. Below is an example job script:

Example Job Script#

Kestrel CPU Sample Submission Script 
#!/bin/bash

# This job requests 102 tasks per node. This may need to be adjusted based on system hardware. 

#SBATCH --time=12:00:00
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=96
#SBATCH --partition=<partition-name>
#SBATCH --exclusive=user
#SBATCH --account=<account-name>
#SBATCH --export=ALL
#SBATCH --job-name
#SBATCH --output=out_%j

module load 4.2.2-cray # or module load 4.2.2-intel

# Note that builds with different toolchains may require different modules and environments to be loaded

export OMP_NUM_THREADS=1

srun wrf.exe

To submit the above WRF jobscript named submit_wrf.sh, do sbatch submit_wrf.sh

Supported Versions#

Kestrel
4.2.2

Advanced#

Building Instructions From Source#

Any WRF version can be downloaded here. Pursuant to building WRF, appropriate versions of netcdf, hdf5 and pnetcdf (if parallel netcdf is required) must also be built since they are dependent for building WRF. These dependent supporting softwares may already be offered as modules. If not, then users need to build them first, instructions for which are not provided here.

Building on Kestrel with Cray Toolchain

Copy the configure.wrf file for the Cray toolchain, which can be found at /nopt/nrel/apps/software/wrf/cray/WRF-4.2.2/configure.wrf on Kestrel, into your WRF build dir WRF-4.x.y, and do % cd WRF-4.x.y.

Building on Kestrel with Intel Toolchain

Copy the configure.wrf file for the Intel toolchain, which can be found at /nopt/nrel/apps/software/wrf/intel/WRF-4.2.2/configure.wrf on Kestrel, into your WRF build dir WRF-4.x.y, and do % cd WRF-4.x.y.

Alternatively, the configure.wrf can also be generated by setting the below modules and environments, and executing % ./configure and choosing the appropriate architecture option.

After loading the modules for the appropriate toolchain, supporting netcdf, hdf5, pnetcdf softwares, and setting the netcdf environments, compile the WRF software:

For Cray toolchain: 

% export NETCDF=/opt/cray/pe/netcdf/default/CRAYCLANG/14.0 
% ./compile em_real

For Intel toolchain: 

% export NETCDF=/nopt/nrel/apps/libraries/08-23/spack/opt/spack/linux-rhel8-icelake/intel-2021.10.0/netcdf 
% ./compile em_real

<! ## Troubleshooting

Include known problems and workarounds here, if applicable !>