sup3r.models.conditional_moments.Sup3rCondMom

class Sup3rCondMom(gen_layers, optimizer=None, learning_rate=0.0001, num_par=None, history=None, meta=None, means=None, stdevs=None, default_device=None, name=None)

Bases: AbstractSingleModel, AbstractInterface

Basic Sup3r conditional moments model.

Parameters:

• gen_layers (list | str) – Hidden layers input argument to phygnn.base.CustomNetwork for the generative super resolving model. Can also be a str filepath to a .json config file containing the input layers argument or a .pkl for a saved pre-trained model.

• optimizer (tf.keras.optimizers.Optimizer | dict | None | str) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.

• learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.

• num_par (int | None) – Number of trainable parameters in the model

• history (pd.DataFrame | str | None) – Model training history with “epoch” index, str pointing to a saved history csv file with “epoch” as first column, or None for clean history

• meta (dict | None) – Model meta data that describes how the model was created.

• means (dict | None) – Set of mean values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

• stdevs (dict | None) – Set of stdev values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

• default_device (str | None) – Option for default device placement of model weights. If None and a single GPU exists, that GPU will be the default device. If None and multiple GPUs exist, the CPU will be the default device (this was tested as most efficient given the custom multi-gpu strategy developed in self.run_gradient_descent())

• name (str | None) – Optional name for the model.

Methods

calc_loss(output_true, output_gen, mask)	Calculate the total moment predictor loss
calc_loss_cond_mom(output_true, output_gen, mask)	Calculate the loss of the moment predictor
calc_val_loss(batch_handler, loss_details)	Calculate the validation loss at the current state of model training
dict_to_tensorboard(entry)	Write data to tensorboard log file.
early_stop(history, column[, threshold, n_epoch])	Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.
finish_epoch(epoch, epochs, t0, ...[, extras])	Perform finishing checks after an epoch is done training
generate(low_res[, norm_in, un_norm_out, ...])	Use the generator model to generate high res data from low res input.
get_high_res_exo_input(high_res)	Get exogenous feature data from high_res
get_loss_fun(loss)	Get the initialized loss function class from the sup3r loss library or the tensorflow losses.
get_optimizer_config(optimizer)	Get a config that defines the current model optimizer
get_s_enhance_from_layers()	Compute factor by which model will enhance spatial resolution from layer attributes.
get_single_grad(low_res, hi_res_true, ...[, ...])	Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.
get_t_enhance_from_layers()	Compute factor by which model will enhance temporal resolution from layer attributes.
init_optimizer(optimizer, learning_rate)	Initialize keras optimizer object.
load(model_dir[, verbose])	Load the model with its sub-networks from a previously saved-to output directory.
load_network(model, name)	Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.
load_saved_params(out_dir[, verbose])	Load saved model_params (you need this and the gen+disc models to load a full model).
log_loss_details(loss_details[, level])	Log the loss details to the module logger.
norm_input(low_res)	Normalize low resolution data being input to the generator.
profile_to_tensorboard(name)	Write profile data to tensorboard log file.
run_gradient_descent(low_res, hi_res_true, ...)	Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights
save(out_dir)	Save the model with its sub-networks to a directory.
save_params(out_dir)
seed([s])	Set the random seed for reproducible results.
set_model_params(**kwargs)	Set parameters used for training the model
set_norm_stats(new_means, new_stdevs)	Set the normalization statistics associated with a data batch handler to model attributes.
train(batch_handler, input_resolution, n_epoch)	Train the model on real low res data and real high res data
train_epoch(batch_handler[, multi_gpu])	Train the model for one epoch.
un_norm_output(output)	Un-normalize synthetically generated output data to physical units
update_loss_details(loss_details, new_data, ...)	Update a dictionary of loss_details with loss information from a new batch.
update_optimizer(**kwargs)	Update optimizer by changing current configuration

Attributes

generator	Get the generative model.
generator_weights	Get a list of layer weights and bias terms for the generator model.
history	Model training history DataFrame (None if not yet trained)
hr_exo_features	Get list of high-resolution exogenous filter names the model uses.
hr_out_features	Get the list of high-resolution output feature names that the generative model outputs.
input_dims	Get dimension of model generator input.
input_resolution	'...km', 'temporal': '...min'}.
is_4d	Check if model expects spatial only input
is_5d	Check if model expects spatiotemporal input
lr_features	Get a list of low-resolution features input to the generative model.
means	Get the data normalization mean values.
meta	Get meta data dictionary that defines how the model was created
model_params	Model parameters, used to save model to disc
optimizer	Get the tensorflow optimizer to perform gradient descent calculations for the generative network.
output_resolution	Resolution of output data.
s_enhance	Factor by which model will enhance spatial resolution.
smoothed_features	Get the list of smoothed input feature names that the generative model was trained on.
smoothing	Value of smoothing parameter used in gaussian filtering of coarsened high res data.
stdevs	Get the data normalization standard deviation values.
t_enhance	Factor by which model will enhance temporal resolution.
total_batches	Record of total number of batches for logging.
version_record	A record of important versions that this model was built with.
weights	Get a list of all the layer weights and bias terms for the generator network

save(out_dir)

Save the model with its sub-networks to a directory.

Parameters:

out_dir (str) – Directory to save model files. This directory will be created if it does not already exist.

classmethod load(model_dir, verbose=True)

Load the model with its sub-networks from a previously saved-to output directory.

Parameters:

• model_dir (str) – Directory to load model files from.

• verbose (bool) – Flag to log information about the loaded model.

Returns:

out (BaseModel) – Returns a pretrained gan model that was previously saved to out_dir

update_optimizer(**kwargs)

Update optimizer by changing current configuration

Parameters:

kwargs (dict) – kwargs to use for optimizer configuration update

property meta

Get meta data dictionary that defines how the model was created

property model_params

Model parameters, used to save model to disc

Returns:

dict

property weights

Get a list of all the layer weights and bias terms for the generator network

calc_loss_cond_mom(output_true, output_gen, mask)

Calculate the loss of the moment predictor

Parameters:

• output_true (tf.Tensor) – True realization output

• output_gen (tf.Tensor) – Predicted realization output

• mask (tf.Tensor) – Mask to apply

Returns:

loss (tf.Tensor) – 0D tensor generator model loss for the MSE loss of the moment predictor

calc_loss(output_true, output_gen, mask)

Calculate the total moment predictor loss

Parameters:

• output_true (tf.Tensor) – True realization output

• output_gen (tf.Tensor) – Predicted realization output

• mask (tf.Tensor) – Mask to apply

Returns:

• loss (tf.Tensor) – 0D tensor representing the loss value for the moment predictor

• loss_details (dict) – Namespace of the breakdown of loss components

calc_val_loss(batch_handler, loss_details)

Calculate the validation loss at the current state of model training

Parameters:

• batch_handler (sup3r.data_handling.preprocessing.BatchHandler) – BatchHandler object to iterate through

• loss_details (dict) – Namespace of the breakdown of loss components

Returns:

loss_details (dict) – Same as input but now includes val_* loss info

dict_to_tensorboard(entry)

Write data to tensorboard log file. This is usually a loss_details dictionary.

Parameters:

entry (dict) – Dictionary of values to write to tensorboard log file

static early_stop(history, column, threshold=0.005, n_epoch=5)

Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.

Parameters:

• history (pd.DataFrame | None) – Model training history

• column (str) – Column from the model training history to evaluate for early termination.

• threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.

• n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.

Returns:

stop (bool) – Flag to stop training (True) or keep going (False).

finish_epoch(epoch, epochs, t0, loss_details, checkpoint_int, out_dir, early_stop_on, early_stop_threshold, early_stop_n_epoch, extras=None)

Perform finishing checks after an epoch is done training

Parameters:

• epoch (int) – Epoch number that is finishing

• epochs (list) – List of epochs being iterated through

• t0 (float) – Starting time of training.

• loss_details (dict) – Namespace of the breakdown of loss components

• checkpoint_int (int | None) – Epoch interval at which to save checkpoint models.

• out_dir (str) – Directory to save checkpoint models. Should have {epoch} in the directory name. This directory will be created if it does not already exist.

• early_stop_on (str | None) – If not None, this should be a column in the training history to evaluate for early stopping (e.g. validation_loss_gen, validation_loss_disc). If this value in this history decreases by an absolute fractional relative difference of less than 0.01 for more than 5 epochs in a row, the training will stop early.

• early_stop_threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.

• early_stop_n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.

• extras (dict | None) – Extra kwargs/parameters to save in the epoch history.

Returns:

stop (bool) – Flag to early stop training.

generate(low_res, norm_in=True, un_norm_out=True, exogenous_data=None)

Use the generator model to generate high res data from low res input. This is the public generate function.

Parameters:

• low_res (np.ndarray) – Low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

• norm_in (bool) – Flag to normalize low_res input data if the self._means, self._stdevs attributes are available. The generator should always received normalized data with mean=0 stdev=1. This also normalizes hi_res_topo.

• un_norm_out (bool) – Flag to un-normalize synthetically generated output data to physical units

• exogenous_data (dict | ExoData | None) – Special dictionary (class:ExoData) of exogenous feature data with entries describing whether features should be combined at input, a mid network layer, or with output. This doesn’t have to include the ‘model’ key since this data is for a single step model.

Returns:

hi_res (ndarray) – Synthetically generated high-resolution data, usually a 4D or 5D array with shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

property generator

Get the generative model.

Returns:

phygnn.base.CustomNetwork

property generator_weights

Get a list of layer weights and bias terms for the generator model.

Returns:

list

get_high_res_exo_input(high_res)

Get exogenous feature data from high_res

Parameters:

high_res (tf.Tensor) – Ground truth high resolution spatiotemporal data.

Returns:

exo_data (dict) – Dictionary of exogenous feature data used as input to tf_generate. e.g. {‘topography’: tf.Tensor(…)}

static get_loss_fun(loss)

Get the initialized loss function class from the sup3r loss library or the tensorflow losses.

Parameters:

loss (str | dict) – Loss function class name from sup3r.utilities.loss_metrics (prioritized) or tensorflow.keras.losses. Defaults to tf.keras.losses.MeanSquaredError. This can be provided as a dict with kwargs for loss functions with extra parameters. e.g. {‘SpatialExtremesLoss’: {‘weight’: 0.5}}

Returns:

out (tf.keras.losses.Loss) – Initialized loss function class that is callable, e.g. if “MeanSquaredError” is requested, this will return an instance of tf.keras.losses.MeanSquaredError()

static get_optimizer_config(optimizer)

Get a config that defines the current model optimizer

Parameters:

optimizer (tf.keras.optimizers.Optimizer) – TF-Keras optimizer object

Returns:

config (dict) – Optimizer config

get_s_enhance_from_layers()

Compute factor by which model will enhance spatial resolution from layer attributes. Used in model training during high res coarsening

get_single_grad(low_res, hi_res_true, training_weights, device_name=None, **calc_loss_kwargs)

Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.

Parameters:

• low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.

• device_name (None | str) – Optional tensorflow device name for GPU placement. Note that if a GPU is available, variables will be placed on that GPU even if device_name=None.

• calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method

Returns:

• grad (list) – a list or nested structure of Tensors (or IndexedSlices, or None, or CompositeTensor) representing the gradients for the training_weights

• loss_details (dict) – Namespace of the breakdown of loss components

get_t_enhance_from_layers()

Compute factor by which model will enhance temporal resolution from layer attributes. Used in model training during high res coarsening

property history

Model training history DataFrame (None if not yet trained)

Returns:

pandas.DataFrame | None

property hr_exo_features

Get list of high-resolution exogenous filter names the model uses. If the model has N concat or add layers this list will be the last N features in the training features list. The ordering is assumed to be the same as the order of concat or add layers. If training features is […, topo, sza], and the model has 2 concat or add layers, exo features will be [topo, sza]. Topo will then be used in the first concat layer and sza will be used in the second

property hr_out_features

Get the list of high-resolution output feature names that the generative model outputs.

static init_optimizer(optimizer, learning_rate)

Initialize keras optimizer object.

Parameters:

• optimizer (tf.keras.optimizers.Optimizer | dict | None | str) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.

• learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.

Returns:

optimizer (tf.keras.optimizers.Optimizer) – Initialized optimizer object.

property input_dims

Get dimension of model generator input. This is usually 4D for spatial models and 5D for spatiotemporal models. This gives the input to the first step if the model is multi-step. Returns 5 for linear models.

Returns:

int

property input_resolution

‘…km’, ‘temporal’: ‘…min’}. The numbers are required to be integers in the units specified. The units are not strict as long as the resolution of the exogenous data, when extracting exogenous data, is specified in the same units.

Type:

Resolution of input data. Given as a dictionary {‘spatial’

property is_4d

Check if model expects spatial only input

property is_5d

Check if model expects spatiotemporal input

load_network(model, name)

Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.

Parameters:

• model (str | dict) – Model hidden layers config, a .json with “hidden_layers” key, or a .pkl for a saved pre-trained model.

• name (str) – Name of the model to be loaded

Returns:

model (phygnn.CustomNetwork) – CustomNetwork object initialized from the model input.

static load_saved_params(out_dir, verbose=True)

Load saved model_params (you need this and the gen+disc models to load a full model).

Parameters:

• out_dir (str) – Directory to load model files from.

• verbose (bool) – Flag to log information about the loaded model.

Returns:

params (dict) – Model parameters loaded from disk json file. This should be the same as self.model_params with and additional ‘history’ entry. Should be all the kwargs you need to init a model.

static log_loss_details(loss_details, level='INFO')

Log the loss details to the module logger.

Parameters:

• loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.

• level (str) – Log level (e.g. INFO, DEBUG)

property lr_features

Get a list of low-resolution features input to the generative model. This includes low-resolution features that might be supplied exogenously at inference time but that were in the low-res batches during training

property means

Get the data normalization mean values.

Returns:

np.ndarray

norm_input(low_res)

Normalize low resolution data being input to the generator.

Parameters:

low_res (np.ndarray) – Un-normalized low-resolution input data in physical units, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

Returns:

low_res (np.ndarray) – Normalized low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)

property optimizer

Get the tensorflow optimizer to perform gradient descent calculations for the generative network. This is functionally identical to optimizer_disc is no special optimizer model or learning rate was specified for the disc.

Returns:

tf.keras.optimizers.Optimizer

property output_resolution

Resolution of output data. Given as a dictionary {‘spatial’: ‘…km’, ‘temporal’: ‘…min’}. This is computed from the input resolution and the enhancement factors.

profile_to_tensorboard(name)

Write profile data to tensorboard log file.

Parameters:

name (str) – Tag name to use for profile info

run_gradient_descent(low_res, hi_res_true, training_weights, optimizer=None, multi_gpu=False, **calc_loss_kwargs)

Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights

Parameters:

• low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)

• training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.

• optimizer (tf.keras.optimizers.Optimizer) – Optimizer class to use to update weights. This can be different if you’re training just the generator or one of the discriminator models. Defaults to the generator optimizer.

• multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and the resulting gradient from each GPU will constitute a single gradient descent step with the nominal learning rate that the model was initialized with.

• calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method

Returns:

loss_details (dict) – Namespace of the breakdown of loss components

property s_enhance

Factor by which model will enhance spatial resolution. Used in model training during high res coarsening

save_params(out_dir)

Parameters:

out_dir (str) – Directory to save linear model params. This directory will be created if it does not already exist.

static seed(s=0)

Set the random seed for reproducible results.

Parameters:

s (int) – Random seed

set_model_params(**kwargs)

Set parameters used for training the model

Parameters:

kwargs (dict) – Keyword arguments including ‘input_resolution’, ‘lr_features’, ‘hr_exo_features’, ‘hr_out_features’, ‘smoothed_features’, ‘s_enhance’, ‘t_enhance’, ‘smoothing’

set_norm_stats(new_means, new_stdevs)

Set the normalization statistics associated with a data batch handler to model attributes.

Parameters:

• new_means (dict | None) – Set of mean values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

• new_stdevs (dict | None) – Set of stdev values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.

property smoothed_features

Get the list of smoothed input feature names that the generative model was trained on.

property smoothing

Value of smoothing parameter used in gaussian filtering of coarsened high res data.

property stdevs

Get the data normalization standard deviation values.

Returns:

np.ndarray

property t_enhance

Factor by which model will enhance temporal resolution. Used in model training during high res coarsening

property total_batches

Record of total number of batches for logging.

un_norm_output(output)

Un-normalize synthetically generated output data to physical units

Parameters:

output (tf.Tensor | np.ndarray) – Synthetically generated high-resolution data

Returns:

output (np.ndarray) – Synthetically generated high-resolution data

static update_loss_details(loss_details, new_data, batch_len, prefix=None)

Update a dictionary of loss_details with loss information from a new batch.

Parameters:

• loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.

• new_data (dict) – Namespace of the breakdown of loss components for a single new batch.

• batch_len (int) – Length of the incoming batch.

• prefix (None | str) – Option to prefix the names of the loss data when saving to the loss_details dictionary.

Returns:

loss_details (dict) – Same as input loss_details but with running averages updated.

property version_record

A record of important versions that this model was built with.

Returns:

dict

train_epoch(batch_handler, multi_gpu=False)

Train the model for one epoch.

Parameters:

• batch_handler (sup3r.data_handling.preprocessing.BatchHandler) – BatchHandler object to iterate through

• multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and the resulting gradient from each GPU will constitute a single gradient descent step with the nominal learning rate that the model was initialized with.

Returns:

loss_details (dict) – Namespace of the breakdown of loss components

train(batch_handler, input_resolution, n_epoch, checkpoint_int=None, out_dir='./condMom_{epoch}', early_stop_on=None, early_stop_threshold=0.005, early_stop_n_epoch=5, multi_gpu=False, tensorboard_log=True)

Train the model on real low res data and real high res data

Parameters:

• batch_handler (sup3r.data_handling.preprocessing.BatchHandler) – BatchHandler object to iterate through

• input_resolution (dict) – Dictionary specifying spatiotemporal input resolution. e.g. {‘temporal’: ‘60min’, ‘spatial’: ‘30km’}

• n_epoch (int) – Number of epochs to train on

• checkpoint_int (int | None) – Epoch interval at which to save checkpoint models.

• out_dir (str) – Directory to save checkpoint models. Should have {epoch} in the directory name. This directory will be created if it does not already exist.

• early_stop_on (str | None) – If not None, this should be a column in the training history to evaluate for early stopping (e.g. validation_loss_gen). If this value in this history decreases by an absolute fractional relative difference of less than 0.01 for more than 5 epochs in a row, the training will stop early.

• early_stop_threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.

• early_stop_n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.

• multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and the resulting gradient from each GPU will constitute a single gradient descent step with the nominal learning rate that the model was initialized with.

• tensorboard_log (bool) – Whether to write log file for use with tensorboard. Log data can be viewed with tensorboard --logdir <logdir> where <logdir> is the parent directory of out_dir, and pointing the browser to the printed address.