sup3r.models.conditional.Sup3rCondMom#
- class Sup3rCondMom(gen_layers, optimizer=None, learning_rate=0.0001, num_par=None, history=None, meta=None, means=None, stdevs=None, default_device=None, name=None)[source]#
Bases:
AbstractSingleModel
,AbstractInterface
Basic Sup3r conditional moments model.
- Parameters:
gen_layers (list | str) – Hidden layers input argument to phygnn.base.CustomNetwork for the generative super resolving model. Can also be a str filepath to a .json config file containing the input layers argument or a .pkl for a saved pre-trained model.
optimizer (tf.keras.optimizers.Optimizer | dict | None | str) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.
learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.
num_par (int | None) – Number of trainable parameters in the model
history (pd.DataFrame | str | None) – Model training history with “epoch” index, str pointing to a saved history csv file with “epoch” as first column, or None for clean history
meta (dict | None) – Model meta data that describes how the model was created.
means (dict | None) – Set of mean values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.
stdevs (dict | None) – Set of stdev values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.
default_device (str | None) – Option for default device placement of model weights. If None and a single GPU exists, that GPU will be the default device. If None and multiple GPUs exist, the CPU will be the default device (this was tested as most efficient given the custom multi-gpu strategy developed in self.run_gradient_descent())
name (str | None) – Optional name for the model.
Methods
calc_loss
(output_true, output_gen, mask)Calculate the total moment predictor loss
calc_loss_cond_mom
(output_true, output_gen, mask)Calculate the loss of the moment predictor
calc_val_loss
(batch_handler, loss_details)Calculate the validation loss at the current state of model training
dict_to_tensorboard
(entry)Write data to tensorboard log file.
early_stop
(history, column[, threshold, n_epoch])Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.
finish_epoch
(epoch, epochs, t0, ...[, extras])Perform finishing checks after an epoch is done training
generate
(low_res[, norm_in, un_norm_out, ...])Use the generator model to generate high res data from low res input.
get_high_res_exo_input
(high_res)Get exogenous feature data from high_res
get_loss_fun
(loss)Get the initialized loss function class from the sup3r loss library or the tensorflow losses.
get_optimizer_config
(optimizer)Get a config that defines the current model optimizer
get_optimizer_state
(optimizer)Get a set of state variables for the optimizer
Compute factor by which model will enhance spatial resolution from layer attributes.
get_single_grad
(low_res, hi_res_true, ...[, ...])Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.
Compute factor by which model will enhance temporal resolution from layer attributes.
init_optimizer
(optimizer, learning_rate)Initialize keras optimizer object.
load
(model_dir[, verbose])Load the model with its sub-networks from a previously saved-to output directory.
load_network
(model, name)Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.
load_saved_params
(out_dir[, verbose])Load saved model_params (you need this and the gen+disc models to load a full model).
log_loss_details
(loss_details[, level])Log the loss details to the module logger.
norm_input
(low_res)Normalize low resolution data being input to the generator.
profile_to_tensorboard
(name)Write profile data to tensorboard log file.
run_gradient_descent
(low_res, hi_res_true, ...)Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights
save
(out_dir)Save the model with its sub-networks to a directory.
save_params
(out_dir)seed
([s])Set the random seed for reproducible results.
set_model_params
(**kwargs)Set parameters used for training the model
set_norm_stats
(new_means, new_stdevs)Set the normalization statistics associated with a data batch handler to model attributes.
train
(batch_handler, input_resolution, n_epoch)Train the model on real low res data and real high res data
train_epoch
(batch_handler[, multi_gpu])Train the model for one epoch.
un_norm_output
(output)Un-normalize synthetically generated output data to physical units
update_loss_details
(loss_details, new_data, ...)Update a dictionary of loss_details with loss information from a new batch.
update_optimizer
(**kwargs)Update optimizer by changing current configuration
Attributes
Get the generative model.
Get a list of layer weights and bias terms for the generator model.
Model training history DataFrame (None if not yet trained)
Get list of high-resolution exogenous filter names the model uses.
Get the list of high-resolution output feature names that the generative model outputs.
Get dimension of model generator input.
Resolution of input data.
Check if model expects spatial only input
Check if model expects spatiotemporal input
Get a list of low-resolution features input to the generative model.
Get the data normalization mean values.
Get meta data dictionary that defines how the model was created
Model parameters, used to save model to disc
Get the tensorflow optimizer to perform gradient descent calculations for the generative network.
Resolution of output data.
Factor by which model will enhance spatial resolution.
List of spatial enhancement factors.
Get the list of smoothed input feature names that the generative model was trained on.
Value of smoothing parameter used in gaussian filtering of coarsened high res data.
Get the data normalization standard deviation values.
Factor by which model will enhance temporal resolution.
List of temporal enhancement factors.
Record of total number of batches for logging.
A record of important versions that this model was built with.
Get a list of all the layer weights and bias terms for the generator network
- save(out_dir)[source]#
Save the model with its sub-networks to a directory.
- Parameters:
out_dir (str) – Directory to save model files. This directory will be created if it does not already exist.
- classmethod load(model_dir, verbose=True)[source]#
Load the model with its sub-networks from a previously saved-to output directory.
- Parameters:
model_dir (str) – Directory to load model files from.
verbose (bool) – Flag to log information about the loaded model.
- Returns:
out (BaseModel) – Returns a pretrained gan model that was previously saved to out_dir
- update_optimizer(**kwargs)[source]#
Update optimizer by changing current configuration
- Parameters:
kwargs (dict) – kwargs to use for optimizer configuration update
- property meta#
Get meta data dictionary that defines how the model was created
- property model_params#
Model parameters, used to save model to disc
- Returns:
dict
- property weights#
Get a list of all the layer weights and bias terms for the generator network
- calc_loss_cond_mom(output_true, output_gen, mask)[source]#
Calculate the loss of the moment predictor
- Parameters:
output_true (tf.Tensor) – True realization output
output_gen (tf.Tensor) – Predicted realization output
mask (tf.Tensor) – Mask to apply
- Returns:
loss (tf.Tensor) – 0D tensor generator model loss for the MSE loss of the moment predictor
- calc_loss(output_true, output_gen, mask)[source]#
Calculate the total moment predictor loss
- Parameters:
output_true (tf.Tensor) – True realization output
output_gen (tf.Tensor) – Predicted realization output
mask (tf.Tensor) – Mask to apply
- Returns:
loss (tf.Tensor) – 0D tensor representing the loss value for the moment predictor
loss_details (dict) – Namespace of the breakdown of loss components
- calc_val_loss(batch_handler, loss_details)[source]#
Calculate the validation loss at the current state of model training
- Parameters:
batch_handler (sup3r.preprocessing.BatchHandler) – BatchHandler object to iterate through
loss_details (dict) – Namespace of the breakdown of loss components
- Returns:
loss_details (dict) – Same as input but now includes val_* loss info
- dict_to_tensorboard(entry)#
Write data to tensorboard log file. This is usually a loss_details dictionary.
- Parameters:
entry (dict) – Dictionary of values to write to tensorboard log file
- static early_stop(history, column, threshold=0.005, n_epoch=5)#
Determine whether to stop training early based on nearly no change to validation loss for a certain number of consecutive epochs.
- Parameters:
history (pd.DataFrame | None) – Model training history
column (str) – Column from the model training history to evaluate for early termination.
threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.
n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.
- Returns:
stop (bool) – Flag to stop training (True) or keep going (False).
- finish_epoch(epoch, epochs, t0, loss_details, checkpoint_int, out_dir, early_stop_on, early_stop_threshold, early_stop_n_epoch, extras=None)#
Perform finishing checks after an epoch is done training
- Parameters:
epoch (int) – Epoch number that is finishing
epochs (list) – List of epochs being iterated through
t0 (float) – Starting time of training.
loss_details (dict) – Namespace of the breakdown of loss components
checkpoint_int (int | None) – Epoch interval at which to save checkpoint models.
out_dir (str) – Directory to save checkpoint models. Should have {epoch} in the directory name. This directory will be created if it does not already exist.
early_stop_on (str | None) – If not None, this should be a column in the training history to evaluate for early stopping (e.g. validation_loss_gen, validation_loss_disc). If this value in this history decreases by an absolute fractional relative difference of less than 0.01 for more than 5 epochs in a row, the training will stop early.
early_stop_threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.
early_stop_n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.
extras (dict | None) – Extra kwargs/parameters to save in the epoch history.
- Returns:
stop (bool) – Flag to early stop training.
- generate(low_res, norm_in=True, un_norm_out=True, exogenous_data=None)#
Use the generator model to generate high res data from low res input. This is the public generate function.
- Parameters:
low_res (np.ndarray) – Low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)
norm_in (bool) – Flag to normalize low_res input data if the self._means, self._stdevs attributes are available. The generator should always received normalized data with mean=0 stdev=1. This also normalizes hi_res_topo.
un_norm_out (bool) – Flag to un-normalize synthetically generated output data to physical units
exogenous_data (dict | ExoData | None) – Special dictionary (class:ExoData) of exogenous feature data with entries describing whether features should be combined at input, a mid network layer, or with output. This doesn’t have to include the ‘model’ key since this data is for a single step model.
- Returns:
hi_res (ndarray) – Synthetically generated high-resolution data, usually a 4D or 5D array with shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)
- property generator#
Get the generative model.
- Returns:
phygnn.base.CustomNetwork
- property generator_weights#
Get a list of layer weights and bias terms for the generator model.
- Returns:
list
- get_high_res_exo_input(high_res)#
Get exogenous feature data from high_res
- Parameters:
high_res (tf.Tensor) – Ground truth high resolution spatiotemporal data.
- Returns:
exo_data (dict) – Dictionary of exogenous feature data used as input to tf_generate. e.g.
{'topography': tf.Tensor(...)}
- static get_loss_fun(loss)#
Get the initialized loss function class from the sup3r loss library or the tensorflow losses.
- Parameters:
loss (str | dict) – Loss function class name from sup3r.utilities.loss_metrics (prioritized) or tensorflow.keras.losses. Defaults to tf.keras.losses.MeanSquaredError. This can be provided as a dict with kwargs for loss functions with extra parameters. e.g. {‘SpatialExtremesLoss’: {‘weight’: 0.5}}
- Returns:
out (tf.keras.losses.Loss) – Initialized loss function class that is callable, e.g. if “MeanSquaredError” is requested, this will return an instance of tf.keras.losses.MeanSquaredError()
- static get_optimizer_config(optimizer)#
Get a config that defines the current model optimizer
- Parameters:
optimizer (tf.keras.optimizers.Optimizer) – TF-Keras optimizer object (e.g., Adam)
- Returns:
config (dict) – Optimizer config
- classmethod get_optimizer_state(optimizer)#
Get a set of state variables for the optimizer
- Parameters:
optimizer (tf.keras.optimizers.Optimizer) – TF-Keras optimizer object (e.g., Adam)
- Returns:
state (dict) – Optimizer state variables
- get_s_enhance_from_layers()#
Compute factor by which model will enhance spatial resolution from layer attributes. Used in model training during high res coarsening
- get_single_grad(low_res, hi_res_true, training_weights, device_name=None, **calc_loss_kwargs)#
Run gradient descent for one mini-batch of (low_res, hi_res_true), do not update weights, just return gradient details.
- Parameters:
low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)
hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)
training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.
device_name (None | str) – Optional tensorflow device name for GPU placement. Note that if a GPU is available, variables will be placed on that GPU even if device_name=None.
calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method
- Returns:
grad (list) – a list or nested structure of Tensors (or IndexedSlices, or None, or CompositeTensor) representing the gradients for the training_weights
loss_details (dict) – Namespace of the breakdown of loss components
- get_t_enhance_from_layers()#
Compute factor by which model will enhance temporal resolution from layer attributes. Used in model training during high res coarsening
- property history#
Model training history DataFrame (None if not yet trained)
- Returns:
pandas.DataFrame | None
- property hr_exo_features#
Get list of high-resolution exogenous filter names the model uses. If the model has N concat or add layers this list will be the last N features in the training features list. The ordering is assumed to be the same as the order of concat or add layers. If training features is […, topo, sza], and the model has 2 concat or add layers, exo features will be [topo, sza]. Topo will then be used in the first concat layer and sza will be used in the second
- property hr_out_features#
Get the list of high-resolution output feature names that the generative model outputs.
- static init_optimizer(optimizer, learning_rate)#
Initialize keras optimizer object.
- Parameters:
optimizer (tf.keras.optimizers.Optimizer | dict | None | str) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.
learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.
- Returns:
optimizer (tf.keras.optimizers.Optimizer) – Initialized optimizer object.
- property input_dims#
Get dimension of model generator input. This is usually 4D for spatial models and 5D for spatiotemporal models. This gives the input to the first step if the model is multi-step. Returns 5 for linear models.
- Returns:
int
- property input_resolution#
Resolution of input data. Given as a dictionary
{'spatial': '...km', 'temporal': '...min'}
. The numbers are required to be integers in the units specified. The units are not strict as long as the resolution of the exogenous data, when extracting exogenous data, is specified in the same units.
- property is_4d#
Check if model expects spatial only input
- property is_5d#
Check if model expects spatiotemporal input
- load_network(model, name)#
Load a CustomNetwork object from hidden layers config, .json file config, or .pkl file saved pre-trained model.
- Parameters:
model (str | dict) – Model hidden layers config, a .json with “hidden_layers” key, or a .pkl for a saved pre-trained model.
name (str) – Name of the model to be loaded
- Returns:
model (phygnn.CustomNetwork) – CustomNetwork object initialized from the model input.
- static load_saved_params(out_dir, verbose=True)#
Load saved model_params (you need this and the gen+disc models to load a full model).
- Parameters:
out_dir (str) – Directory to load model files from.
verbose (bool) – Flag to log information about the loaded model.
- Returns:
params (dict) – Model parameters loaded from disk json file. This should be the same as self.model_params with and additional ‘history’ entry. Should be all the kwargs you need to init a model.
- static log_loss_details(loss_details, level='INFO')#
Log the loss details to the module logger.
- Parameters:
loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.
level (str) – Log level (e.g. INFO, DEBUG)
- property lr_features#
Get a list of low-resolution features input to the generative model. This includes low-resolution features that might be supplied exogenously at inference time but that were in the low-res batches during training
- property means#
Get the data normalization mean values.
- Returns:
np.ndarray
- norm_input(low_res)#
Normalize low resolution data being input to the generator.
- Parameters:
low_res (np.ndarray) – Un-normalized low-resolution input data in physical units, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)
- Returns:
low_res (np.ndarray) – Normalized low-resolution input data, usually a 4D or 5D array of shape: (n_obs, spatial_1, spatial_2, n_features) (n_obs, spatial_1, spatial_2, n_temporal, n_features)
- property optimizer#
Get the tensorflow optimizer to perform gradient descent calculations for the generative network. This is functionally identical to optimizer_disc is no special optimizer model or learning rate was specified for the disc.
- Returns:
tf.keras.optimizers.Optimizer
- property output_resolution#
Resolution of output data. Given as a dictionary {‘spatial’: ‘…km’, ‘temporal’: ‘…min’}. This is computed from the input resolution and the enhancement factors.
- profile_to_tensorboard(name)#
Write profile data to tensorboard log file.
- Parameters:
name (str) – Tag name to use for profile info
- run_gradient_descent(low_res, hi_res_true, training_weights, optimizer=None, multi_gpu=False, **calc_loss_kwargs)#
Run gradient descent for one mini-batch of (low_res, hi_res_true) and update weights
- Parameters:
low_res (np.ndarray) – Real low-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)
hi_res_true (np.ndarray) – Real high-resolution data in a 4D or 5D array: (n_observations, spatial_1, spatial_2, features) (n_observations, spatial_1, spatial_2, temporal, features)
training_weights (list) – A list of layer weights that are to-be-trained based on the current loss weight values.
optimizer (tf.keras.optimizers.Optimizer) – Optimizer class to use to update weights. This can be different if you’re training just the generator or one of the discriminator models. Defaults to the generator optimizer.
multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and resulting gradients from each GPU will be summed and then applied once per batch at the nominal learning rate that the model and optimizer were initialized with.
calc_loss_kwargs (dict) – Kwargs to pass to the self.calc_loss() method
- Returns:
loss_details (dict) – Namespace of the breakdown of loss components
- property s_enhance#
Factor by which model will enhance spatial resolution. Used in model training during high res coarsening and also in forward pass routine to determine shape of needed exogenous data
- property s_enhancements#
List of spatial enhancement factors. In the case of a single step model this is just
[self.s_enhance]
. This is used to determine shapes of needed exogenous data in forward pass routine
- save_params(out_dir)#
- Parameters:
out_dir (str) – Directory to save linear model params. This directory will be created if it does not already exist.
- static seed(s=0)#
Set the random seed for reproducible results.
- Parameters:
s (int) – Random seed
- set_model_params(**kwargs)#
Set parameters used for training the model
- Parameters:
kwargs (dict) – Keyword arguments including ‘input_resolution’, ‘lr_features’, ‘hr_exo_features’, ‘hr_out_features’, ‘smoothed_features’, ‘s_enhance’, ‘t_enhance’, ‘smoothing’
- set_norm_stats(new_means, new_stdevs)#
Set the normalization statistics associated with a data batch handler to model attributes.
- Parameters:
new_means (dict | None) – Set of mean values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.
new_stdevs (dict | None) – Set of stdev values for data normalization keyed by feature name. Can be used to maintain a consistent normalization scheme between transfer learning domains.
- property smoothed_features#
Get the list of smoothed input feature names that the generative model was trained on.
- property smoothing#
Value of smoothing parameter used in gaussian filtering of coarsened high res data.
- property stdevs#
Get the data normalization standard deviation values.
- Returns:
np.ndarray
- property t_enhance#
Factor by which model will enhance temporal resolution. Used in model training during high res coarsening and also in forward pass routine to determine shape of needed exogenous data
- property t_enhancements#
List of temporal enhancement factors. In the case of a single step model this is just
[self.t_enhance]
. This is used to determine shapes of needed exogenous data in forward pass routine
- property total_batches#
Record of total number of batches for logging.
- un_norm_output(output)#
Un-normalize synthetically generated output data to physical units
- Parameters:
output (tf.Tensor | np.ndarray) – Synthetically generated high-resolution data
- Returns:
output (np.ndarray) – Synthetically generated high-resolution data
- static update_loss_details(loss_details, new_data, batch_len, prefix=None)#
Update a dictionary of loss_details with loss information from a new batch.
- Parameters:
loss_details (dict) – Namespace of the breakdown of loss components where each value is a running average at the current state in the epoch.
new_data (dict) – Namespace of the breakdown of loss components for a single new batch.
batch_len (int) – Length of the incoming batch.
prefix (None | str) – Option to prefix the names of the loss data when saving to the loss_details dictionary.
- Returns:
loss_details (dict) – Same as input loss_details but with running averages updated.
- property version_record#
A record of important versions that this model was built with.
- Returns:
dict
- train_epoch(batch_handler, multi_gpu=False)[source]#
Train the model for one epoch.
- Parameters:
batch_handler (sup3r.preprocessing.BatchHandler) – BatchHandler object to iterate through
multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and the resulting gradient from each GPU will constitute a single gradient descent step with the nominal learning rate that the model was initialized with.
- Returns:
loss_details (dict) – Namespace of the breakdown of loss components
- train(batch_handler, input_resolution, n_epoch, checkpoint_int=None, out_dir='./condMom_{epoch}', early_stop_on=None, early_stop_threshold=0.005, early_stop_n_epoch=5, multi_gpu=False, tensorboard_log=True)[source]#
Train the model on real low res data and real high res data
- Parameters:
batch_handler (sup3r.preprocessing.BatchHandler) – BatchHandler object to iterate through
input_resolution (dict) – Dictionary specifying spatiotemporal input resolution. e.g. {‘temporal’: ‘60min’, ‘spatial’: ‘30km’}
n_epoch (int) – Number of epochs to train on
checkpoint_int (int | None) – Epoch interval at which to save checkpoint models.
out_dir (str) – Directory to save checkpoint models. Should have {epoch} in the directory name. This directory will be created if it does not already exist.
early_stop_on (str | None) – If not None, this should be a column in the training history to evaluate for early stopping (e.g. validation_loss_gen). If this value in this history decreases by an absolute fractional relative difference of less than 0.01 for more than 5 epochs in a row, the training will stop early.
early_stop_threshold (float) – The absolute relative fractional difference in validation loss between subsequent epochs below which an early termination is warranted. E.g. if val losses were 0.1 and 0.0998 the relative diff would be calculated as 0.0002 / 0.1 = 0.002 which would be less than the default thresold of 0.01 and would satisfy the condition for early termination.
early_stop_n_epoch (int) – The number of consecutive epochs that satisfy the threshold that warrants an early stop.
multi_gpu (bool) – Flag to break up the batch for parallel gradient descent calculations on multiple gpus. If True and multiple GPUs are present, each batch from the batch_handler will be divided up between the GPUs and the resulting gradient from each GPU will constitute a single gradient descent step with the nominal learning rate that the model was initialized with.
tensorboard_log (bool) – Whether to write log file for use with tensorboard. Log data can be viewed with
tensorboard --logdir <logdir>
where<logdir>
is the parent directory ofout_dir
, and pointing the browser to the printed address.