sup3r.pipeline.slicer.ForwardPassSlicer

class ForwardPassSlicer(coarse_shape: tuple | list, time_steps: int, s_enhance: int, t_enhance: int, time_slice: slice, temporal_pad: int, spatial_pad: int, chunk_shape: tuple | list)

Bases: object

Get slices for sending data chunks through generator.

Parameters:

• coarse_shape (tuple) – Shape of full domain for low res data

• time_steps (int) – Number of time steps for full temporal domain of low res data. This is used to construct a dummy_time_index from np.arange(time_steps)

• time_slice (slice | list) – Slice to use to extract range from time_index. Can be a slice(start, stop, step) or list [start, stop, step]

• chunk_shape (tuple) – Max shape (spatial_1, spatial_2, temporal) of an unpadded coarse chunk to use for a forward pass. The number of nodes that the ForwardPassStrategy is set to distribute to is calculated by dividing up the total time index from all file_paths by the temporal part of this chunk shape. Each node will then be parallelized accross parallel processes by the spatial chunk shape. If temporal_pad / spatial_pad are non zero the chunk sent to the generator can be bigger than this shape. If running in serial set this equal to the shape of the full spatiotemporal data volume for best performance.

• s_enhance (int) – Spatial enhancement factor

• t_enhance (int) – Temporal enhancement factor

• spatial_pad (int) – Size of spatial overlap between coarse chunks passed to forward passes for subsequent spatial stitching. This overlap will pad both sides of the fwp_chunk_shape. Note that the first and last chunks in any of the spatial dimension will not be padded.

• temporal_pad (int) – Size of temporal overlap between coarse chunks passed to forward passes for subsequent temporal stitching. This overlap will pad both sides of the fwp_chunk_shape. Note that the first and last chunks in the temporal dimension will not be padded.

Methods

get_cropped_slices(unpadded_slices, ...)	Get cropped slices to cut off padded output
get_hr_slices(slices, enhancement[, step])	Get high resolution slices for temporal or spatial slices
get_padded_slices(slices, shape, ...[, step])	Get padded slices with the specified padding size, max shape, enhancement, and step size
get_spatial_slices()	Get spatial slices for small data chunks that are passed through generator
get_time_slices()	Calculate the number of time chunks across the full time index

Attributes

chunk_lookup	Get a 3D array with shape (n_spatial_1_chunks, n_spatial_2_chunks, n_time_chunks) where each value is the chunk index.
hr_crop_slices	Get high res spatiotemporal cropped slices for cropping generator output
n_chunks	Get total number of spatiotemporal chunks
n_spatial_chunks	Get the number of spatial chunks
n_time_chunks	Get the number of temporal chunks
s1_hr_slices	Get high res spatial slices for first spatial dimension
s1_lr_pad_slices	List of low resolution spatial slices with padding for first spatial dimension
s1_lr_slices	List of low resolution spatial slices for first spatial dimension considering padding on all sides of the spatial raster.
s2_hr_slices	Get high res spatial slices for second spatial dimension
s2_lr_pad_slices	List of low resolution spatial slices with padding for second spatial dimension
s2_lr_slices	List of low resolution spatial slices for second spatial dimension considering padding on all sides of the spatial raster.
s_hr_crop_slices	Get high res cropped slices for cropping generator output
s_hr_slices	Get high res slices for indexing full generator output array
s_lr_crop_slices	Get low res cropped slices for cropping input chunk domain
s_lr_pad_slices	Get low res padded slices for small data chunks that are passed through generator
s_lr_slices	Get low res spatial slices for small data chunks that are passed through generator
spatial_chunk_lookup	Get a 2D array with shape (n_spatial_1_chunks, n_spatial_2_chunks) where each value is the spatial chunk index.
t_hr_crop_slices	Get high res temporal cropped slices for cropping forward pass output before stitching together
t_lr_crop_slices	Get low res temporal cropped slices for cropping time index of padded input data.
t_lr_pad_slices	Get low res temporal padded slices for distributing time chunks across nodes.
t_lr_slices	Low resolution temporal slices
coarse_shape
time_steps
s_enhance
t_enhance
time_slice
temporal_pad
spatial_pad
chunk_shape

get_spatial_slices()

Get spatial slices for small data chunks that are passed through generator

Returns:

• s_lr_slices (list) – List of slices for low res data chunks which have not been padded. data_handler.data[s_lr_slice] corresponds to an unpadded low res input to the model.

• s_lr_pad_slices (list) – List of slices which have been padded so that high res output can be stitched together. data_handler.data[s_lr_pad_slice] corresponds to a padded low res input to the model.

• s_hr_slices (list) – List of slices for high res data corresponding to the lr_slices regions. output_array[s_hr_slice] corresponds to the cropped generator output.

get_time_slices()

Calculate the number of time chunks across the full time index

Returns:

• t_lr_slices (list) – List of low-res non-padded time index slices. e.g. If fwp_chunk_size[2] is 5 then the size of these slices will always be 5.

• t_lr_pad_slices (list) – List of low-res padded time index slices. e.g. If fwp_chunk_size[2] is 5 the size of these slices will be 15, with exceptions at the start and end of the full time index.

property s_lr_slices

Get low res spatial slices for small data chunks that are passed through generator

Returns:

_s_lr_slices (list) – List of spatial slices corresponding to the unpadded spatial region going through the generator

property s_lr_pad_slices

Get low res padded slices for small data chunks that are passed through generator

Returns:

_s_lr_pad_slices (list) – List of slices which have been padded so that high res output can be stitched together. Each entry in this list has a slice for each spatial dimension. data_handler.data[s_lr_pad_slice] gives the padded data volume passed through the generator

property t_lr_pad_slices

Get low res temporal padded slices for distributing time chunks across nodes. These slices correspond to the time chunks sent to each node and are padded according to temporal_pad.

Returns:

_t_lr_pad_slices (list) – List of low res temporal slices which have been padded so that high res output can be stitched together

property t_lr_crop_slices

Get low res temporal cropped slices for cropping time index of padded input data.

Returns:

_t_lr_crop_slices (list) – List of low res temporal slices for cropping padded input data

property t_hr_crop_slices

Get high res temporal cropped slices for cropping forward pass output before stitching together

Returns:

_t_hr_crop_slices (list) – List of high res temporal slices for cropping padded generator output

property s1_hr_slices

Get high res spatial slices for first spatial dimension

property s2_hr_slices

Get high res spatial slices for second spatial dimension

property s_hr_slices

Get high res slices for indexing full generator output array

Returns:

_s_hr_slices (list) – List of high res slices. Each entry in this list has a slice for each spatial dimension. output[hr_slice] gives the superresolved domain corresponding to data_handler.data[lr_slice]

property s_lr_crop_slices

Get low res cropped slices for cropping input chunk domain

Returns:

_s_lr_crop_slices (list) – List of low res cropped slices. Each entry in this list has a slice for each spatial dimension.

property s_hr_crop_slices

Get high res cropped slices for cropping generator output

Returns:

_s_hr_crop_slices (list) – List of high res cropped slices. Each entry in this list has a slice for each spatial dimension.

property hr_crop_slices

Get high res spatiotemporal cropped slices for cropping generator output

Returns:

_hr_crop_slices (list) – List of high res spatiotemporal cropped slices. Each entry in this list has a crop slice for each spatial dimension and temporal dimension and then slice(None) for the feature dimension. model.generate()[hr_crop_slice] gives the cropped generator output corresponding to outpuUnion[np.ndarray, da.core.Array][hr_slice]

property s1_lr_pad_slices

List of low resolution spatial slices with padding for first spatial dimension

property s2_lr_pad_slices

List of low resolution spatial slices with padding for second spatial dimension

property s1_lr_slices

List of low resolution spatial slices for first spatial dimension considering padding on all sides of the spatial raster.

property s2_lr_slices

List of low resolution spatial slices for second spatial dimension considering padding on all sides of the spatial raster.

property t_lr_slices

Low resolution temporal slices

static get_hr_slices(slices, enhancement, step=None)

Get high resolution slices for temporal or spatial slices

Parameters:

• slices (list) – Low resolution slices to be enhanced

• enhancement (int) – Enhancement factor

• step (int | None) – Step size for slices

Returns:

hr_slices (list) – High resolution slices

property chunk_lookup

Get a 3D array with shape (n_spatial_1_chunks, n_spatial_2_chunks, n_time_chunks) where each value is the chunk index.

property spatial_chunk_lookup

Get a 2D array with shape (n_spatial_1_chunks, n_spatial_2_chunks) where each value is the spatial chunk index.

property n_spatial_chunks

Get the number of spatial chunks

property n_time_chunks

Get the number of temporal chunks

property n_chunks

Get total number of spatiotemporal chunks

static get_padded_slices(slices, shape, enhancement, padding, step=None)

Get padded slices with the specified padding size, max shape, enhancement, and step size

Parameters:

• slices (list) – List of low res unpadded slice

• shape (int) – max possible index of a padded slice. e.g. if the slices are indexing a dimension with size 10 then a padded slice cannot have an index greater than 10.

• enhancement (int) – Enhancement factor. e.g. If these slices are indexing a spatial dimension which will be enhanced by 2x then enhancement=2.

• padding (int) – Padding factor. e.g. If these slices are indexing a spatial dimension and the spatial_pad is 10 this is 10. It will be multiplied by the enhancement factor if the slices are to be used to index an enhanced dimension.

• step (int | None) – Step size for slices. e.g. If these slices are indexing a temporal dimension and time_slice.step = 3 then step=3.

Returns:

list – Padded slices for temporal or spatial dimensions.

static get_cropped_slices(unpadded_slices, padded_slices, enhancement)

Get cropped slices to cut off padded output

Parameters:

• unpadded_slices (list) – List of unpadded slices

• padded_slices (list) – List of padded slices

• enhancement (int) – Enhancement factor for the data to be cropped.

Returns:

list – Cropped slices for temporal or spatial dimensions.