sup3r.preprocessing.data_handlers.exo.ExoDataHandler

class ExoDataHandler(file_paths: str | list | Path, feature: str, model: Sup3rGan | MultiStepGan | None = None, steps: list | None = None, source_file: str | None = None, input_handler_name: str | None = None, input_handler_kwargs: dict | None = None, cache_dir: str = './exo_cache', chunks: str | dict | None = 'auto', distance_upper_bound: int | None = None)

Bases: object

Class to rasterize exogenous features for multistep forward passes. e.g. Multiple topography arrays at different resolutions for multiple spatial enhancement steps.

This takes a list of models and uses the different sets of models features to retrieve and rasterize exogenous data according to the requested target coordinate and grid shape, for each model step.

Parameters:

• file_paths (str | list) – A single source h5 file or netcdf file to extract raster data from. The string can be a unix-style file path which will be passed through glob.glob. This is typically low-res WRF output or GCM netcdf data that is source low-resolution data intended to be sup3r resolved.

• feature (str) – Exogenous feature to extract from file_paths

• model (Sup3rGan | MultiStepGan) – Model used to get exogenous data. If a MultiStepGan lr_features, hr_exo_features, and hr_out_features will be checked for each model in model.models and exogenous data will be retrieved based on the resolution required for that type of feature. e.g. If a model has topography as a lr and hr_exo feature, and the model performs 5x spatial enhancement with an input resolution of 30km then topography at 30km and at 6km will be retrieved. Either this or list of steps needs to be provided.

• steps (list) – List of dictionaries containing info on which models to use for a given step index and what type of exo data the step requires. e.g.:: [{‘model’: 0, ‘combine_type’: ‘input’},

  {‘model’: 0, ‘combine_type’: ‘layer’}]

  Each step entry can also contain enhancement factors. e.g.:: [{‘model’: 0, ‘combine_type’: ‘input’, ‘s_enhance’: 1, ‘t_enhance’: 1},

  {‘model’: 0, ‘combine_type’: ‘layer’, ‘s_enhance’: 3, ‘t_enhance’: 1}]

• source_file (str) – Filepath to source wtk, nsrdb, or netcdf file to get hi-res data from which will be mapped to the enhanced grid of the file_paths input. Pixels from this file will be mapped to their nearest low-res pixel in the file_paths input. Accordingly, the input should be a significantly higher resolution than file_paths. Warnings will be raised if the low-resolution pixels in file_paths do not have unique nearest pixels from this exo source data.

• input_handler_name (str) – data handler class used by the exo handler. Provide a string name to match a Rasterizer. If None the correct handler will be guessed based on file type and time series properties. This is passed directly to the exo handler, along with input_handler_kwargs

• input_handler_kwargs (dict | None) – Any kwargs for initializing the input_handler_name class used by the exo handler.

• cache_dir (str | None) – Directory for storing cache data. Default is ‘./exo_cache’. If None then no data will be cached.

• chunks (str | dict) – Dictionary of dimension chunk sizes for returned exo data. e.g. {‘time’: 100, ‘south_north’: 100, ‘west_east’: 100}. This can also just be “auto”. This is passed to .chunk() before returning exo data through .data attribute

• distance_upper_bound (float | None) – Maximum distance to map high-resolution data from source_file to the low-resolution file_paths input. None (default) will calculate this based on the median distance between points in source_file

Methods

get_all_step_data()	Get exo data for each model step.
get_exo_rasterizer(s_enhance, t_enhance)	Get exo rasterizer instance for given enhancement factors
get_exo_steps(feature, models)	Get list of steps describing how to use exogenous data for the given feature in the list of given models.
get_single_step_data(s_enhance, t_enhance)	Get exo data for a single model step, with specific enhancement factors.

Attributes

cache_dir
cache_files	Get exo data cache file for all enhancement factors
chunks
distance_upper_bound
input_handler_kwargs
input_handler_name
model
source_file
steps
file_paths
feature

classmethod get_exo_steps(feature, models)

Get list of steps describing how to use exogenous data for the given feature in the list of given models. This checks the input and exo feature lists for each model step and adds that step if the given feature is found in the list.

get_exo_rasterizer(s_enhance, t_enhance)

Get exo rasterizer instance for given enhancement factors

get_single_step_data(s_enhance, t_enhance)

Get exo data for a single model step, with specific enhancement factors.

property cache_files

Get exo data cache file for all enhancement factors

get_all_step_data()

Get exo data for each model step.