sup3r.preprocessing.dual_batch_handling.DualValidationData
- class DualValidationData(data_handlers, batch_size=8, s_enhance=1, t_enhance=1, temporal_coarsening_method='subsample', hr_features_ind=None, smoothing=None, smoothing_ignore=None)[source]
Bases:
ValidationData
Iterator for validation data for training with dual data handler
- Parameters:
data_handlers (list[DataHandler]) – List of DataHandler instances
batch_size (int) – Size of validation data batches
s_enhance (int) – Factor by which to coarsen spatial dimensions of the high resolution data
t_enhance (int) – Factor by which to coarsen temporal dimension of the high resolution data
temporal_coarsening_method (str) – [subsample, average, total, min, max] Subsample will take every t_enhance-th time step, average will average over t_enhance time steps, total will sum over t_enhance time steps
hr_features_ind (list | np.ndarray | None) – List/array of feature channel indices that are used for generative output, without any feature indices used only for training.
smoothing (float | None) – Standard deviation to use for gaussian filtering of the coarse data. This can be tuned by matching the kinetic energy of a low resolution simulation with the kinetic energy of a coarsened and smoothed high resolution simulation. If None no smoothing is performed.
smoothing_ignore (list | None) – List of features to ignore for the smoothing filter. None will smooth all features if smoothing kwarg is not None
Methods
any
()Return True if any validation data exists
batch_next
(high_res)Assemble the next batch
Get random handler index based on handler weights
Attributes
Get weights used to sample from different data handlers based on relative sizes
Get sample shape for high_res data
Get sample shape for low_res data
Shape of full validation dataset across all handlers
- property shape
Shape of full validation dataset across all handlers
- Returns:
shape (tuple) – (spatial_1, spatial_2, temporal, features) With temporal extent equal to the sum across all data handlers time dimension
- property hr_sample_shape
Get sample shape for high_res data
- property lr_sample_shape
Get sample shape for low_res data
- any()
Return True if any validation data exists
- batch_next(high_res)
Assemble the next batch
- Parameters:
high_res (np.ndarray) – 4D | 5D array (batch_size, spatial_1, spatial_2, features) (batch_size, spatial_1, spatial_2, temporal, features)
- Returns:
batch (Batch)
- get_handler_index()
Get random handler index based on handler weights
- property handler_weights
Get weights used to sample from different data handlers based on relative sizes