sup3r.postprocessing.writers.h5.OutputHandlerH5

sup3r.postprocessing.writers.h5.OutputHandlerH5#

class OutputHandlerH5[source]#

Bases: OutputHandler

Class to handle writing output to H5 file

Methods

enforce_limits(features, data)

Enforce physical limits for feature data

get_dset_attrs(feature)

Get attrributes for output feature

get_lat_lon(low_res_lat_lon, shape)

Get lat lon arrays for high res output file

get_renamed_features(features)

Rename features based on transformation from u/v to windspeed/winddirection

get_time_dim_name(filepath)

Get the name of the time dimension in the given file

get_times(low_res_times, shape)

Get array of times for high res output file

invert_uv_features(data, features, lat_lon)

Invert U/V to windspeed and winddirection.

invert_uv_single_pair(data, lat_lon, u_idx, ...)

Perform inverse transform in place on a single u/v pair.

is_increasing_lons(lat_lon)

Check if longitudes are in increasing order.

pad_lat_lon(lat_lon)

Pad lat lon grid with additional rows and columns to use for interpolation routine

write_data(out_file, dsets, time_index, ...)

Write list of datasets to out_file.

write_output(data, features, ...[, ...])

Write forward pass output to file
static enforce_limits(features, data)#

Enforce physical limits for feature data

Parameters:

  • features (list) – List of features with ordering corresponding to last channel of data array.

  • data (ndarray) – Array of feature data

Returns:

data (ndarray) – Array of feature data with physical limits enforced

static get_dset_attrs(feature)#

Get attrributes for output feature

Parameters:

feature (str) – Name of feature to write

Returns:

  • attrs (dict) – Dictionary of attributes for requested dset

  • dtype (str) – Data type for requested dset. Defaults to float32

classmethod get_lat_lon(low_res_lat_lon, shape)#

Get lat lon arrays for high res output file

Parameters:

  • low_res_lat_lon (ndarray) – Array of lat/lon for input data. Longitudes must be arranged in a counter-clockwise direction (when looking down from above the north pole). e.g. [-50, -25, 25, 50] or [130, 180, -130, -80]. The latter passes through the 180 -> -180 boundary and will be temporarily shifted to the 0-360 range before interpolation. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

  • shape (tuple) – (lons, lats) Shape of high res grid

Returns:

lat_lon (ndarray) – Array of lat lons for high res output file (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

classmethod get_renamed_features(features)#

Rename features based on transformation from u/v to windspeed/winddirection

Parameters:

features (list) – List of output features

Returns:

list – List of renamed features u/v -> windspeed/winddirection for each height

static get_time_dim_name(filepath)#

Get the name of the time dimension in the given file

Parameters:

filepath (str) – Path to the file

Returns:

time_key (str) – Name of the time dimension in the given file

static get_times(low_res_times, shape)#

Get array of times for high res output file

Parameters:

  • low_res_times (pd.Datetimeindex) – List of times for low res input data. If there is only a single low res timestep, it is assumed the data is daily.

  • shape (int) – Number of time steps for high res time array

Returns:

ndarray – Array of times for high res output file.

classmethod invert_uv_features(data, features, lat_lon, max_workers=None)#

Invert U/V to windspeed and winddirection. Performed in place.

Parameters:

  • data (ndarray) – High res data from forward pass (spatial_1, spatial_2, temporal, features)

  • features (list) – List of output features. If this doesn’t contain any names matching u_*m, this method will do nothing.

  • lat_lon (ndarray) – High res lat/lon array (spatial_1, spatial_2, 2)

  • max_workers (int | None) – Max workers to use for inverse transform. If None the maximum possible will be used

static invert_uv_single_pair(data, lat_lon, u_idx, v_idx)#

Perform inverse transform in place on a single u/v pair.

Parameters:

  • data (ndarray) – High res data from forward pass (spatial_1, spatial_2, temporal, features)

  • lat_lon (ndarray) – High res lat/lon array (spatial_1, spatial_2, 2)

  • u_idx (int) – Index in data for U component to transform

  • v_idx (int) – Index in data for V component to transform

static is_increasing_lons(lat_lon)#

Check if longitudes are in increasing order. Need to check this for interpolation routine. This is primarily to identify whether the lons go through the 180 -> -180 boundary, which creates a discontinuity. For example, [130, 180, -130, -80]. If any lons go from positive to negative the lons need to be shifted to the range 0-360.

Parameters:

lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

Returns:

bool – Whether all lons are in increasing order or not

static pad_lat_lon(lat_lon)#

Pad lat lon grid with additional rows and columns to use for interpolation routine

Parameters:

  • lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

  • shape (tuple) – (lons, lats) Shape of high res grid

Returns:

lat_lon (ndarray) – Array of padded lat lons (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

classmethod write_data(out_file, dsets, time_index, data_list, meta, global_attrs=None)#

Write list of datasets to out_file.

Parameters:

  • out_file (str) – Pre-existing H5 file output path

  • dsets (list) – list of datasets to write to out_file

  • time_index (pd.DatetimeIndex()) – Pandas datetime index to use for file time_index.

  • data_list (list) – List of np.ndarray objects to write to out_file

  • meta (pd.DataFrame) – Full meta dataframe for the final output data.

  • global_attrs (dict) – Namespace of file-global attributes for the final output data.

classmethod write_output(data, features, low_res_lat_lon, low_res_times, out_file, meta_data=None, invert_uv=None, max_workers=None, gids=None)#

Write forward pass output to file

Parameters:

  • data (ndarray) – (spatial_1, spatial_2, temporal, features) High resolution forward pass output

  • features (list) – List of feature names corresponding to the last dimension of data

  • low_res_lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)

  • low_res_times (pd.Datetimeindex) – List of times for low res source data

  • out_file (string) – Output file path

  • meta_data (dict | None) – Dictionary of meta data from model

  • invert_uv (bool | None) – Whether to convert u and v wind components to windspeed and direction

  • max_workers (int | None) – Max workers to use for inverse uv transform. If None the max_workers will be estimated based on memory limits.

  • gids (list) – List of coordinate indices used to label each lat lon pair and to help with spatial chunk data collection