sup3r.preprocessing.data_handlers.exo.ExoData

class ExoData(steps)

Bases: dict

Special dictionary class for multiple exogenous_data steps

TODO: Can we simplify this by relying more on xr.Dataset meta data instead of storing enhancement factors for each step? Seems like we could take the highest res data and coarsen based on s/t enhance, also.

Combine multiple SingleExoDataStep objects

Parameters:

steps (dict) – Dictionary with feature keys each with entries describing whether features should be combined at input, a mid network layer, or with output. e.g.:

‘topography’: {

‘steps’: [

{‘combine_type’: ‘input’, ‘model’: 0, ‘data’: …}, {‘combine_type’: ‘layer’, ‘model’: 0, ‘data’: …}]

}

}

Each array in in ‘data’ key has 3D or 4D shape: (spatial_1, spatial_2, 1) (spatial_1, spatial_2, n_temporal, 1)

Methods

clear()
copy()
fromkeys([value])	Create a new dictionary with keys from iterable and values set to value.
get(key[, default])	Return the value for key if key is in the dictionary, else default.
get_chunk(lr_slices)	Get the data for all model steps corresponding to the low res extent selected by lr_slices
get_combine_type_data(feature, combine_type)	Get exogenous data for given feature which is used according to the given combine_type (input/output/layer) for this model_step.
get_model_step_exo(model_step)	Get the exogenous data for the given model_step from the full list of steps
items()
keys()
pop(k[,d])	If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem()	Remove and return a (key, value) pair as a 2-tuple.
setdefault(key[, default])	Insert key with a value of default if key is not in the dictionary.
split(split_steps)	Split self into multiple ExoData objects based on split_steps.
update([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values()

get_model_step_exo(model_step)

Get the exogenous data for the given model_step from the full list of steps

Parameters:

model_step (int) – Index of the model to get exogenous data for.

Returns:

model_step_exo (dict) – Dictionary of features each with list of steps which match the given model_step

split(split_steps)

Split self into multiple ExoData objects based on split_steps. The splits are done such that the steps in the ith entry of the returned list all have a model number < split_steps[i].

Note

This is used for multi-step models to correctly distribute the set of all exo data steps to the appropriate models. For example, MultiStepGan models with some temporal (spatial) steps followed by some spatial (temporal) steps. The temporal (spatial) models might take the first N exo data steps and then the spatial (temporal) models will take the remaining exo data steps.

Parameters:

split_steps (list) – Step index list to use for splitting. To split this into exo data for spatial models and temporal models split_steps should be [len(spatial_models)]. If this is for a MultiStepGan model with temporal steps first, split_steps should be [len(temporal_models)]. If this is for a multi step model composed of more than two models (e.g. SolarMultiStepGan) split_steps should be [len(spatial_solar_models), len(spatial_solar_models) + len(spatial_wind_models)]

Returns:

split_list (List[ExoData]) – List of ExoData objects coming from the split of self, according to split_steps

get_combine_type_data(feature, combine_type, model_step=None)

Get exogenous data for given feature which is used according to the given combine_type (input/output/layer) for this model_step.

Parameters:

• feature (str) – Name of exogenous feature to get data for

• combine_type (str) – Usage type for requested data. e.g input/output/layer

• model_step (int | None) – Model step the data will be used for. If this is not None then only steps with self[feature][‘steps’][:][‘model’] == model_step will be searched for data.

Returns:

data (tf.Tensor | np.ndarray) – Exogenous data for given parameters

get_chunk(lr_slices)

Get the data for all model steps corresponding to the low res extent selected by lr_slices

Parameters:

• lr_slices (list List of spatiotemporal slices which specify extent of)

• the low-resolution input data.

Returns:

exo_data (ExoData) – ExoData object composed of multiple SingleExoDataStep objects. This is the sliced exo data for the extent specified by lr_slices.

clear() → None.  Remove all items from D.

copy() → a shallow copy of D

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values