sup3r.utilities.pytest.helpers.DummySampler

class DummySampler(sample_shape, data_shape, features, batch_size, feature_sets=None)

Bases: Sampler

Dummy container with random data.

Parameters:

• data (Union[Sup3rX, Sup3rDataset],) – Object with data that will be sampled from. Usually the .data attribute of various Container objects. i.e. Loader, Rasterizer, Deriver, as long as the spatial dimensions are not flattened.

• sample_shape (tuple) – Size of arrays to sample from the contained data.

• batch_size (int) – Number of samples to get to build a single batch. A sample of (sample_shape[0], sample_shape[1], batch_size * sample_shape[2]) is first selected from underlying dataset and then reshaped into (batch_size, *sample_shape) to get a single batch. This is more efficient than getting N = batch_size samples and then stacking.

• feature_sets (Optional[dict]) – Optional dictionary describing how the full set of features is split between lr_only_features and hr_exo_features.

  featureslist | tuple

  List of full set of features to use for sampling. If no entry is provided then all data_vars from data will be used.

  lr_only_featureslist | tuple

  List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.

  hr_exo_featureslist | tuple

  List of feature names or patt*erns that should be included in the high-resolution observation but not expected to be output from the generative model. An example is high-res topography that is to be injected mid-network.

Methods

get_sample_index([n_obs])	Randomly gets spatiotemporal sample index.
post_init_log([args_dict])	Log additional arguments after initialization.
preflight()	Check if the sample_shape is larger than the requested raster size
wrap(data)	Return a Sup3rDataset object or tuple of such.

Attributes

data	Return underlying data.
hr_exo_features	Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection.
hr_features	Get the high-resolution features corresponding to hr_features_ind
hr_features_ind	Get the high-resolution feature channel indices that should be included for training.
hr_out_features	Get a list of high-resolution features that are intended to be output by the GAN.
hr_sample_shape	Shape of the data sample to select when __next__() is called.
lr_only_features	List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.
sample_shape	Shape of the data sample to select when __next__() is called.
shape	Get shape of underlying data.

property data

Return underlying data.

Returns:

Sup3rDataset

See also

wrap()

get_sample_index(n_obs=None)

Randomly gets spatiotemporal sample index.

Notes

If n_obs > 1 this will get a time slice with n_obs * self.sample_shape[2] time steps, which will then be reshaped into n_obs samples each with self.sample_shape[2] time steps. This is a much more efficient way of getting batches of samples but only works if there are enough continuous time steps to sample.

Returns:

sample_index (tuple) – Tuple of latitude slice, longitude slice, time slice, and features. Used to get single observation like self.data[sample_index]

property hr_exo_features

Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection. These must come at the end of the high-res feature set. These can also be input to the model as low-res features.

property hr_features

Get the high-resolution features corresponding to hr_features_ind

property hr_features_ind

Get the high-resolution feature channel indices that should be included for training. Any high-resolution features that are only included in the data handler to be coarsened for the low-res input are removed

property hr_out_features

Get a list of high-resolution features that are intended to be output by the GAN. Does not include high-resolution exogenous features

property hr_sample_shape: Tuple

Shape of the data sample to select when __next__() is called. Same as sample_shape

property lr_only_features

List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.

post_init_log(args_dict=None)

Log additional arguments after initialization.

preflight()

Check if the sample_shape is larger than the requested raster size

property sample_shape: Tuple

Shape of the data sample to select when __next__() is called.

property shape

Get shape of underlying data.

wrap(data)

Return a Sup3rDataset object or tuple of such. This is a tuple when the .data attribute belongs to a Collection object like BatchHandler. Otherwise this is Sup3rDataset object, which is either a wrapped 2-tuple or 1-tuple (e.g. len(data) == 2 or len(data) == 1). This is a 2-tuple when .data belongs to a dual container object like DualSampler and a 1-tuple otherwise.