sup3r.models.abstract.AbstractSingleModel

class AbstractSingleModel

Bases: ABC, TensorboardMixIn

Abstract class to define the required training interface for Sup3r model subclasses

Methods

dict_to_tensorboard(entry)	Write data to tensorboard log file.
early_stop(history, column[, threshold, n_epoch])	Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.
finish_epoch(epoch, epochs, t0, ...[, extras])	Perform finishing checks after an epoch is done training
generate(low_res[, norm_in, un_norm_out, ...])	Use the generator model to generate high res data from low res input.
get_high_res_exo_input(high_res)	Get exogenous feature data from high_res
get_loss_fun(loss)	Get the initialized loss function class from the sup3r loss library or the tensorflow losses.
get_optimizer_config(optimizer)	Get a config that defines the current model optimizer
get_optimizer_state(optimizer)	Get a set of state variables for the optimizer
get_single_grad(low_res, hi_res_true, ...[, ...])	Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.
init_optimizer(optimizer, learning_rate)	Initialize keras optimizer object.
load_network(model, name)	Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.
load_saved_params(out_dir[, verbose])	Load saved model_params (you need this and the gen+disc models to load a full model).
log_loss_details(loss_details[, level])	Log the loss details to the module logger.
norm_input(low_res)	Normalize low resolution data being input to the generator.
profile_to_tensorboard(name)	Write profile data to tensorboard log file.
run_gradient_descent(low_res, hi_res_true, ...)	Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights
save(out_dir)	Save the model with its sub-networks to a directory.
set_norm_stats(new_means, new_stdevs)	Set the normalization statistics associated with a data batch handler to model attributes.
un_norm_output(output)	Un-normalize synthetically generated output data to physical units
update_loss_details(loss_details, new_data, ...)	Update a dictionary of loss_details with loss information from a new batch.

Attributes

generator	Get the generative model.
generator_weights	Get a list of layer weights and bias terms for the generator model.
history	Model training history DataFrame (None if not yet trained)
means	Get the data normalization mean values.
optimizer	Get the tensorflow optimizer to perform gradient descent calculations for the generative network.
stdevs	Get the data normalization standard deviation values.
total_batches	Record of total number of batches for logging.

load_network(model, name)

Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.

Parameters:

• model (str | dict) – Model hidden layers config, a .json with “hidden_layers” key, or a .pkl for a saved pre-trained model.

• name (str) – Name of the model to be loaded

Returns:

model (phygnn.CustomNetwork) – CustomNetwork object initialized from the model input.

property means

Get the data normalization mean values.

Returns:

np.ndarray

property stdevs

Get the data normalization standard deviation values.

Returns:

np.ndarray

set_norm_stats(new_means, new_stdevs)

Set the normalization statistics associated with a data batch handler to model attributes.

Parameters:

• new_means (dict | None) – Set of mean values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

• new_stdevs (dict | None) – Set of stdev values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

norm_input(low_res)

Normalize low resolution data being input to the generator.

Parameters:

low_res (np.ndarray) – Un-normalized low-resolution input data in physical units, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

Returns:

low_res (np.ndarray) – Normalized low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

un_norm_output(output)

Un-normalize synthetically generated output data to physical units

Parameters:

output (tf.Tensor | np.ndarray) – Synthetically generated high-resolution data

Returns:

output (np.ndarray) – Synthetically generated high-resolution data

property optimizer

Get the tensorflow optimizer to perform gradient descent calculations for the generative network. This is functionally identical to optimizer_disc is no special optimizer model or learning rate was specified for the disc.

Returns:

tf.keras.optimizers.Optimizer

property history

Model training history DataFrame (None if not yet trained)

Returns:

pandas.DataFrame | None

property generator

Get the generative model.

Returns:

phygnn.base.CustomNetwork

property generator_weights

Get a list of layer weights and bias terms for the generator model.

Returns:

list

static init_optimizer(optimizer, learning_rate)

Initialize keras optimizer object.

Parameters:

• optimizer (tf.keras.optimizers.Optimizer | dict | None | str) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.

• learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.

Returns:

optimizer (tf.keras.optimizers.Optimizer) – Initialized optimizer object.

static load_saved_params(out_dir, verbose=True)

Load saved model_params (you need this and the gen+disc models to load a full model).

Parameters:

• out_dir (str) – Directory to load model files from.

• verbose (bool) – Flag to log information about the loaded model.

Returns:

params (dict) – Model parameters loaded from disk json file. This should be the same as self.model_params with and additional ‘history’ entry. Should be all the kwargs you need to init a model.

get_high_res_exo_input(high_res)

Get exogenous feature data from high_res

Parameters:

high_res (tf.Tensor) – Ground truth high resolution spatiotemporal data.

Returns:

exo_data (dict) – Dictionary of exogenous feature data used as input to tf_generate. e.g. {'topography': tf.Tensor(...)}

static get_loss_fun(loss)

Get the initialized loss function class from the sup3r loss library or the tensorflow losses.

Parameters:

loss (str | dict) – Loss function class name from sup3r.utilities.loss_metrics (prioritized) or tensorflow.keras.losses. Defaults to tf.keras.losses.MeanSquaredError. This can be provided as a dict with kwargs for loss functions with extra parameters. e.g. {‘SpatialExtremesLoss’: {‘weight’: 0.5}}

Returns:

out (tf.keras.losses.Loss) – Initialized loss function class that is callable, e.g. if “MeanSquaredError” is requested, this will return an instance of tf.keras.losses.MeanSquaredError()

static get_optimizer_config(optimizer)

Get a config that defines the current model optimizer

Parameters:

optimizer (tf.keras.optimizers.Optimizer) – TF-Keras optimizer object (e.g., Adam)

Returns:

config (dict) – Optimizer config

classmethod get_optimizer_state(optimizer)

Get a set of state variables for the optimizer

Parameters:

optimizer (tf.keras.optimizers.Optimizer) – TF-Keras optimizer object (e.g., Adam)

Returns:

state (dict) – Optimizer state variables

static update_loss_details(loss_details, new_data, batch_len, prefix=None)

Update a dictionary of loss_details with loss information from a new batch.

Parameters:

• loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.

• new_data (dict) – Namespace of the breakdown of loss components for a single new batch.

• batch_len (int) – Length of the incoming batch.

• prefix (None | str) – Option to prefix the names of the loss data when saving to the loss_details dictionary.

Returns:

loss_details (dict) – Same as input loss_details but with running averages updated.

static log_loss_details(loss_details, level='INFO')

Log the loss details to the module logger.

Parameters:

• loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.

• level (str) – Log level (e.g. INFO, DEBUG)

static early_stop(history, column, threshold=0.005, n_epoch=5)

Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.

Parameters:

• history (pd.DataFrame | None) – Model training history

• column (str) – Column from the model training history to evaluate for early termination.

• threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.

• n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.

Returns:

stop (bool) – Flag to stop training (True) or keep going (False).

abstract save(out_dir)

Save the model with its sub-networks to a directory.

Parameters:

out_dir (str) – Directory to save model files. This directory will be created if it does not already exist.

finish_epoch(epoch, epochs, t0, loss_details, checkpoint_int, out_dir, early_stop_on, early_stop_threshold, early_stop_n_epoch, extras=None)

Perform finishing checks after an epoch is done training

Parameters:

• epoch (int) – Epoch number that is finishing

• epochs (list) – List of epochs being iterated through

• t0 (float) – Starting time of training.

• loss_details (dict) – Namespace of the breakdown of loss components

• checkpoint_int (int | None) – Epoch interval at which to save checkpoint models.

• out_dir (str) – Directory to save checkpoint models. Should have {epoch} in the directory name. This directory will be created if it does not already exist.

• early_stop_on (str | None) – If not None, this should be a column in the training history to evaluate for early stopping (e.g. validation_loss_gen, validation_loss_disc). If this value in this history decreases by an absolute fractional relative difference of less than 0.01 for more than 5 epochs in a row, the training will stop early.

• early_stop_threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.

• early_stop_n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.

• extras (dict | None) – Extra kwargs/parameters to save in the epoch history.

Returns:

stop (bool) – Flag to early stop training.

run_gradient_descent(low_res, hi_res_true, training_weights, optimizer=None, multi_gpu=False, **calc_loss_kwargs)

Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights

Parameters:

• low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.

• optimizer (tf.keras.optimizers.Optimizer) – Optimizer class to use to update weights. This can be different if you’re training just the generator or one of the discriminator models. Defaults to the generator optimizer.

• multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and resulting gradients from each GPU will be summed and then applied once per batch at the nominal learning rate that the model and optimizer were initialized with.

• calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method

Returns:

loss_details (dict) – Namespace of the breakdown of loss components

generate(low_res, norm_in=True, un_norm_out=True, exogenous_data=None)

Use the generator model to generate high res data from low res input. This is the public generate function.

Parameters:

• low_res (np.ndarray) – Low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

• norm_in (bool) – Flag to normalize low_res input data if the self._means, self._stdevs attributes are available. The generator should always received normalized data with mean=0 stdev=1. This also normalizes hi_res_topo.

• un_norm_out (bool) – Flag to un-normalize synthetically generated output data to physical units

• exogenous_data (dict | ExoData | None) – Special dictionary (class:ExoData) of exogenous feature data with entries describing whether features should be combined at input, a mid network layer, or with output. This doesn’t have to include the ‘model’ key since this data is for a single step model.

Returns:

hi_res (ndarray) – Synthetically generated high-resolution data, usually a 4D or 5D array with shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

dict_to_tensorboard(entry)

Write data to tensorboard log file. This is usually a loss_details dictionary.

Parameters:

entry (dict) – Dictionary of values to write to tensorboard log file

get_single_grad(low_res, hi_res_true, training_weights, device_name=None, **calc_loss_kwargs)

Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.

Parameters:

• low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.

• device_name (None | str) – Optional tensorflow device name for GPU placement. Note that if a GPU is available, variables will be placed on that GPU even if device_name=None.

• calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method

Returns:

• grad (list) – a list or nested structure of Tensors (or IndexedSlices, or None, or CompositeTensor) representing the gradients for the training_weights

• loss_details (dict) – Namespace of the breakdown of loss components

profile_to_tensorboard(name)

Write profile data to tensorboard log file.

Parameters:

name (str) – Tag name to use for profile info

property total_batches

Record of total number of batches for logging.