sup3r.postprocessing.file_handling.OutputHandlerH5
- class OutputHandlerH5[source]
Bases:
OutputHandler
Class to handle writing output to H5 file
Methods
enforce_limits
(features, data)Enforce physical limits for feature data
get_dset_attrs
(feature)Get attrributes for output feature
get_lat_lon
(low_res_lat_lon, shape)Get lat lon arrays for high res output file
get_renamed_features
(features)Rename features based on transformation from u/v to windspeed/winddirection
get_time_dim_name
(filepath)Get the name of the time dimension in the given file
get_times
(low_res_times, shape)Get array of times for high res output file
invert_uv_features
(data, features, lat_lon)Invert U/V to windspeed and winddirection.
invert_uv_single_pair
(data, lat_lon, u_idx, ...)Perform inverse transform in place on a single u/v pair.
is_increasing_lons
(lat_lon)Check if longitudes are in increasing order.
pad_lat_lon
(lat_lon)Pad lat lon grid with additional rows and columns to use for interpolation routine
write_data
(out_file, dsets, time_index, ...)Write list of datasets to out_file.
write_output
(data, features, ...[, ...])Write forward pass output to file
- classmethod get_renamed_features(features)[source]
Rename features based on transformation from u/v to windspeed/winddirection
- Parameters:
features (list) – List of output features
- Returns:
list – List of renamed features u/v -> windspeed/winddirection for each height
- classmethod invert_uv_features(data, features, lat_lon, max_workers=None)[source]
Invert U/V to windspeed and winddirection. Performed in place.
- Parameters:
data (ndarray) – High res data from forward pass (spatial_1, spatial_2, temporal, features)
features (list) – List of output features. If this doesnt contain any names matching U_*m, this method will do nothing.
lat_lon (ndarray) – High res lat/lon array (spatial_1, spatial_2, 2)
max_workers (int | None) – Max workers to use for inverse transform. If None the max_workers will be estimated based on memory limits.
- static invert_uv_single_pair(data, lat_lon, u_idx, v_idx)[source]
Perform inverse transform in place on a single u/v pair.
- Parameters:
data (ndarray) – High res data from forward pass (spatial_1, spatial_2, temporal, features)
lat_lon (ndarray) – High res lat/lon array (spatial_1, spatial_2, 2)
u_idx (int) – Index in data for U component to transform
v_idx (int) – Index in data for V component to transform
- static enforce_limits(features, data)
Enforce physical limits for feature data
- Parameters:
features (list) – List of features with ordering corresponding to last channel of data array.
data (ndarray) – Array of feature data
- Returns:
data (ndarray) – Array of feature data with physical limits enforced
- static get_dset_attrs(feature)
Get attrributes for output feature
- Parameters:
feature (str) – Name of feature to write
- Returns:
attrs (dict) – Dictionary of attributes for requested dset
dtype (str) – Data type for requested dset. Defaults to float32
- classmethod get_lat_lon(low_res_lat_lon, shape)
Get lat lon arrays for high res output file
- Parameters:
low_res_lat_lon (ndarray) – Array of lat/lon for input data. Longitudes must be arranged in a counter-clockwise direction (when looking down from above the north pole). e.g. [-50, -25, 25, 50] or [130, 180, -130, -80]. The latter passes through the 180 -> -180 boundary and will be temporarily shifted to the 0-360 range before interpolation. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
shape (tuple) – (lons, lats) Shape of high res grid
- Returns:
lat_lon (ndarray) – Array of lat lons for high res output file (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
- static get_time_dim_name(filepath)
Get the name of the time dimension in the given file
- Parameters:
filepath (str) – Path to the file
- Returns:
time_key (str) – Name of the time dimension in the given file
- static get_times(low_res_times, shape)
Get array of times for high res output file
- Parameters:
low_res_times (pd.Datetimeindex) – List of times for low res input data. If there is only a single low res timestep, it is assumed the data is daily.
shape (int) – Number of time steps for high res time array
- Returns:
ndarray – Array of times for high res output file.
- static is_increasing_lons(lat_lon)
Check if longitudes are in increasing order. Need to check this for interpolation routine. This is primarily to identify whether the lons go through the 180 -> -180 boundary, which creates a discontinuity. For example, [130, 180, -130, -80]. If any lons go from positive to negative the lons need to be shifted to the range 0-360.
- Parameters:
lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
- Returns:
bool – Whether all lons are in increasing order or not
- static pad_lat_lon(lat_lon)
Pad lat lon grid with additional rows and columns to use for interpolation routine
- Parameters:
lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
shape (tuple) – (lons, lats) Shape of high res grid
- Returns:
lat_lon (ndarray) – Array of padded lat lons (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
- classmethod write_data(out_file, dsets, time_index, data_list, meta, global_attrs=None)
Write list of datasets to out_file.
- Parameters:
out_file (str) – Pre-existing H5 file output path
dsets (list) – list of datasets to write to out_file
time_index (pd.DatetimeIndex()) – Pandas datetime index to use for file time_index.
data_list (list) – List of np.ndarray objects to write to out_file
meta (pd.DataFrame) – Full meta dataframe for the final output data.
global_attrs (dict) – Namespace of file-global attributes for the final output data.
- classmethod write_output(data, features, low_res_lat_lon, low_res_times, out_file, meta_data=None, max_workers=None, gids=None)
Write forward pass output to file
- Parameters:
data (ndarray) – (spatial_1, spatial_2, temporal, features) High resolution forward pass output
features (list) – List of feature names corresponding to the last dimension of data
low_res_lat_lon (ndarray) – Array of lat/lon for input data. (spatial_1, spatial_2, 2) Last dimension has ordering (lat, lon)
low_res_times (pd.Datetimeindex) – List of times for low res source data
out_file (string) – Output file path
meta_data (dict | None) – Dictionary of meta data from model
max_workers (int | None) – Max workers to use for inverse uv transform. If None the max_workers will be estimated based on memory limits.
gids (list) – List of coordinate indices used to label each lat lon pair and to help with spatial chunk data collection