nsrdb.tmy.tmy_runner.TmyRunner

class TmyRunner(nsrdb_base_fp, years, weights, sites_per_worker=100, n_nodes=1, node_index=0, site_slice=None, out_dir='/tmp/scratch/tmy/', fn_out='tmy.h5', supplemental_fp=None, var_meta=None)[source]

Bases: object

Class to handle running TMY, collecting outs, and writing to files.

Parameters:

  • nsrdb_base_fp (str) – Base nsrdb filepath to retrieve annual files from. Must include a single {} format option for the year. Can include * for an NSRDB multi file source.

  • years (iterable) – Iterable of years to include in the TMY calculation.

  • weights (dict) – Lookup of {dset: weight} where dset is a variable h5 dset name and weight is a fractional TMY weighting. All weights must sum to 1.0

  • sites_per_worker (int) – Number of sites to run at once (sites per core/worker).

  • n_nodes (int) – Number of nodes being run.

  • node_index (int) – Index of this node job.

  • site_slice (slice) – Sites to consider in the GLOBAL TMY run. If multiple jobs are being run, the site slice should be the same for all jobs, and slices the full spatial extent meta data.

  • out_dir (str) – Directory to dump temporary output files.

  • fn_out (str) – Final output filename.

  • supplemental_fp (None | dict) – Supplemental data base filepaths including {} for year for uncommon dataset inputs to the TMY calculation. For example: {‘poa’: ‘/projects/pxs/poa_h5_dir/poa_out_{}.h5’}

  • var_meta (str) – CSV filepath containing meta data for all NSRDB variables. Defaults to the NSRDB var meta csv in git repo.

Methods

collect(nsrdb_base_fp, years, out_dir, fn_out)

Run TMY collection.

get_dset_attrs(dsets[, var_meta])

Get output file dataset attributes for a set of datasets.

run_single(nsrdb_base_fp, years, weights, ...)

Run TMY for a single site chunk (slice) and save to disk.

tmy(nsrdb_base_fp, years, out_dir, fn_out[, ...])

Run the TMY.

Attributes

WEIGHTS

dsets

Get the NSRDB datasets excluding meta and time index.

meta

Get the full NSRDB meta data.

site_chunks

Get a list of site chunk slices to parallelize across
property meta

Get the full NSRDB meta data.

property dsets

Get the NSRDB datasets excluding meta and time index.

property site_chunks

Get a list of site chunk slices to parallelize across

static get_dset_attrs(dsets, var_meta=None)[source]

Get output file dataset attributes for a set of datasets.

Parameters:

  • dsets (list) – List of dataset / variable names.

  • var_meta (str | pd.DataFrame | None) – CSV file or dataframe containing meta data for all NSRDB variables. Defaults to the NSRDB var meta csv in git repo.

Returns:

  • attrs (dict) – Dictionary of dataset attributes keyed by dset name.

  • chunks (dict) – Dictionary of chunk tuples keyed by dset name.

  • dtypes (dict) – dictionary of numpy datatypes keyed by dset name.

static run_single(nsrdb_base_fp, years, weights, site_slice, dsets, f_out, supplemental_fp=None, var_meta=None)[source]

Run TMY for a single site chunk (slice) and save to disk.

Parameters:

  • nsrdb_base_fp (str) – Base nsrdb filepath to retrieve annual files from. Must include a single {} format option for the year. Can include * for an NSRDB multi file source.

  • years (iterable) – Iterable of years to include in the TMY calculation.

  • weights (dict) – Lookup of {dset: weight} where dset is a variable h5 dset name and weight is a fractional TMY weighting. All weights must sum to 1.0

  • site_slice (slice) – Sites to consider in this TMY chunk.

  • dsets (list) – List of TMY datasets to make.

  • f_out (str) – Filepath to save file for this chunk.

  • supplemental_fp (None | dict) – Supplemental data base filepaths including {} for year for uncommon dataset inputs to the TMY calculation. For example: {‘poa’: ‘/projects/pxs/poa_h5_dir/poa_out_{}.h5’}

  • var_meta (str | pd.DataFrame | None) – CSV file or dataframe containing meta data for all NSRDB variables. Defaults to the NSRDB var meta csv in git repo.

Returns:

True

classmethod tmy(nsrdb_base_fp, years, out_dir, fn_out, tmy_type='tmy', weights=None, sites_per_worker=100, n_nodes=1, node_index=0, log=True, log_level='INFO', log_file=None, site_slice=None, supplemental_fp=None, var_meta=None)[source]

Run the TMY. Option for custom weights. Select default weights for TMY / TDY / TGY with tmy_type and weights = None

classmethod collect(nsrdb_base_fp, years, out_dir, fn_out, sites_per_worker=100, site_slice=None, supplemental_fp=None, var_meta=None, log=True, log_level='INFO', log_file=None, purge_chunks=False)[source]

Run TMY collection.

Parameters:

  • nsrdb_base_fp (str) – Base nsrdb filepath to retrieve annual files from. Must include a single {} format option for the year. Can include * for an NSRDB multi file source.

  • years (iterable) – Iterable of years to include in the TMY calculation.

  • out_dir (str) – Directory to dump temporary output files.

  • fn_out (str) – Final output filename.

  • sites_per_worker (int) – Number of sites to run at once (sites per core/worker). Used here to determine size of chunks to collect. Needs to match the value used during the initial call to tmy().

  • site_slice (slice) – Sites to consider in this TMY.

  • supplemental_fp (None | dict) – Supplemental data base filepaths including {} for year for uncommon dataset inputs to the TMY calculation. For example: {‘poa’: ‘/projects/pxs/poa_h5_dir/poa_out_{}.h5’}

  • var_meta (str) – CSV filepath containing meta data for all NSRDB variables. Defaults to the NSRDB var meta csv in git repo.

  • log (bool) – Whether to write to logger

  • log_level (str) – Log level for log output

  • log_file (str | None) – Optional log file to write log output

  • purge_chunks (bool) – Whether to purge chunks after collection into single file.