sup3r.preprocessing.samplers.dc.SamplerDC

class SamplerDC(data: Sup3rX | Sup3rDataset, sample_shape: tuple | None = None, batch_size: int = 16, feature_sets: Dict | None = None, spatial_weights: ndarray | Array | List | None = None, temporal_weights: ndarray | Array | List | None = None)

Bases: Sampler

DataCentric Sampler class used for sampling based on weights which can be updated during training.

Parameters:

• data (Union[Sup3rX, Sup3rDataset],) – Object with data that will be sampled from. Usually the .data attribute of various Container objects. i.e. Loader, Rasterizer, Deriver, as long as the spatial dimensions are not flattened.

• sample_shape (tuple) – Size of arrays to sample from the contained data.

• batch_size (int) – Number of samples to get to build a single batch. A sample of (sample_shape[0], sample_shape[1], batch_size * sample_shape[2]) is first selected from underlying dataset and then reshaped into (batch_size, *sample_shape) to get a single batch. This is more efficient than getting N = batch_size samples and then stacking.

• feature_sets (Optional[dict]) – Optional dictionary describing how the full set of features is split between lr_only_features and hr_exo_features. See Sampler

• spatial_weights (Union[np.ndarray, da.core.Array] | List | None) – Set of weights used to initialize the spatial sampling. e.g. If we want to start off sampling across 2 spatial bins evenly this should be [0.5, 0.5]. During training these weights will be updated based only performance across the bins associated with these weights.

• temporal_weights (Union[np.ndarray, da.core.Array] | List | None) – Set of weights used to initialize the temporal sampling. e.g. If we want to start off sampling only the first season of the year this should be [1, 0, 0, 0]. During training these weights will be updated based only performance across the bins associated with these weights.

Methods

get_sample_index([n_obs])	Randomly gets weighted spatial sample and time sample indices
post_init_log([args_dict])	Log additional arguments after initialization.
preflight()	Check if the sample_shape is larger than the requested raster size
update_weights(spatial_weights, temporal_weights)	Update spatial and temporal sampling weights.
wrap(data)	Return a Sup3rDataset object or tuple of such.

Attributes

data	Return underlying data.
hr_exo_features	Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection.
hr_features	Get the high-resolution features corresponding to hr_features_ind
hr_features_ind	Get the high-resolution feature channel indices that should be included for training.
hr_out_features	Get a list of high-resolution features that are intended to be output by the GAN.
hr_sample_shape	Shape of the data sample to select when __next__() is called.
lr_only_features	List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.
sample_shape	Shape of the data sample to select when __next__() is called.
shape	Get shape of underlying data.

update_weights(spatial_weights, temporal_weights)

Update spatial and temporal sampling weights.

get_sample_index(n_obs=None)

Randomly gets weighted spatial sample and time sample indices

Returns:

observation_index (tuple) – Tuple of sampled spatial grid, time slice, and features indices. Used to get single observation like self.data[observation_index]

property data

Return underlying data.

Returns:

Sup3rDataset

See also

wrap()

property hr_exo_features

Get a list of exogenous high-resolution features that are only used for training e.g., mid-network high-res topo injection. These must come at the end of the high-res feature set. These can also be input to the model as low-res features.

property hr_features

Get the high-resolution features corresponding to hr_features_ind

property hr_features_ind

Get the high-resolution feature channel indices that should be included for training. Any high-resolution features that are only included in the data handler to be coarsened for the low-res input are removed

property hr_out_features

Get a list of high-resolution features that are intended to be output by the GAN. Does not include high-resolution exogenous features

property hr_sample_shape: Tuple

Shape of the data sample to select when __next__() is called. Same as sample_shape

property lr_only_features

List of feature names or patt*erns that should only be included in the low-res training set and not the high-res observations.

post_init_log(args_dict=None)

Log additional arguments after initialization.

preflight()

Check if the sample_shape is larger than the requested raster size

property sample_shape: Tuple

Shape of the data sample to select when __next__() is called.

property shape

Get shape of underlying data.

wrap(data)

Return a Sup3rDataset object or tuple of such. This is a tuple when the .data attribute belongs to a Collection object like BatchHandler. Otherwise this is Sup3rDataset object, which is either a wrapped 2-tuple or 1-tuple (e.g. len(data) == 2 or len(data) == 1). This is a 2-tuple when .data belongs to a dual container object like DualSampler and a 1-tuple otherwise.