sup3r.preprocessing.samplers.dual.DualSampler

class DualSampler(data: Sup3rDataset, sample_shape: tuple | None = None, batch_size: int = 16, s_enhance: int = 1, t_enhance: int = 1, feature_sets: Dict | None = None)

Bases: Sampler

Sampler for sampling from paired (or dual) datasets. Pairs consist of low and high resolution data, which are contained by a Sup3rDataset. This can also include extra observation data on the same grid as the high-resolution data which has NaNs at points where observation data doesn’t exist. This will be used in an additional content loss term.

Parameters:

• data (Sup3rDataset) – A Sup3rDataset instance with low-res and high-res data members, and optionally an obs member.

• sample_shape (tuple) – Size of arrays to sample from the high-res data. The sample shape for the low-res sampler will be determined from the enhancement factors.

• s_enhance (int) – Spatial enhancement factor

• t_enhance (int) – Temporal enhancement factor

• feature_sets (Optional[dict]) – Optional dictionary describing how the full set of features is split between lr_only_features and hr_exo_features.

  lr_only_featureslist | tuple

  List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.

  hr_exo_featureslist | tuple

  List of feature names or patt*erns that should be included in the high-resolution observation but not expected to be output from the generative model. An example is high-res topography that is to be injected mid-network.

Methods

check_for_consistent_shapes()	Make sure container shapes are compatible with enhancement factors.
get_features(feature_sets)	Return default set of features composed from data vars in low res and high res data objects or the value provided through the feature_sets dictionary.
get_sample_index([n_obs])	Get paired sample index, consisting of index for the low res sample and the index for the high res sample with the same spatiotemporal extent.
post_init_log([args_dict])	Log additional arguments after initialization.
preflight()	Check if the sample_shape is larger than the requested raster size
wrap(data)	Return a Sup3rDataset object or tuple of such.

Attributes

data	Return underlying data.
hr_exo_features	Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection.
hr_features	Get the high-resolution features corresponding to hr_features_ind
hr_features_ind	Get the high-resolution feature channel indices that should be included for training.
hr_out_features	Get a list of high-resolution features that are intended to be output by the GAN.
hr_sample_shape	Shape of the data sample to select when __next__() is called.
lr_only_features	List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.
sample_shape	Shape of the data sample to select when __next__() is called.
shape	Get shape of underlying data.

get_features(feature_sets)

Return default set of features composed from data vars in low res and high res data objects or the value provided through the feature_sets dictionary.

check_for_consistent_shapes()

Make sure container shapes are compatible with enhancement factors.

get_sample_index(n_obs=None)

Get paired sample index, consisting of index for the low res sample and the index for the high res sample with the same spatiotemporal extent. Optionally includes an extra high res index if the sample data includes observation data.

property data

Return underlying data.

Returns:

Sup3rDataset

See also

wrap()

property hr_exo_features

Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection. These must come at the end of the high-res feature set. These can also be input to the model as low-res features.

property hr_features

Get the high-resolution features corresponding to hr_features_ind

property hr_features_ind

Get the high-resolution feature channel indices that should be included for training. Any high-resolution features that are only included in the data handler to be coarsened for the low-res input are removed

property hr_out_features

Get a list of high-resolution features that are intended to be output by the GAN. Does not include high-resolution exogenous features

property hr_sample_shape: Tuple

Shape of the data sample to select when __next__() is called. Same as sample_shape

property lr_only_features

List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.

post_init_log(args_dict=None)

Log additional arguments after initialization.

preflight()

Check if the sample_shape is larger than the requested raster size

property sample_shape: Tuple

Shape of the data sample to select when __next__() is called.

property shape

Get shape of underlying data.

wrap(data)

Return a Sup3rDataset object or tuple of such. This is a tuple when the .data attribute belongs to a Collection object like BatchHandler. Otherwise this is Sup3rDataset object, which is either a wrapped 3-tuple, 2-tuple, or 1-tuple (e.g. len(data) == 3, len(data) == 2 or len(data) == 1). This is a 3-tuple when .data belongs to a container object like DualSamplerWithObs, a 2-tuple when .data belongs to a dual container object like DualSampler, and a 1-tuple otherwise.