phygnn.phygnn.PhysicsGuidedNeuralNetwork

class PhysicsGuidedNeuralNetwork(p_fun, loss_weights=(0.5, 0.5), n_features=1, n_labels=1, hidden_layers=None, input_layer=None, output_layer=None, layers_obj=None, metric='mae', optimizer=None, learning_rate=0.01, history=None, kernel_reg_rate=0.0, kernel_reg_power=1, bias_reg_rate=0.0, bias_reg_power=1, feature_names=None, output_names=None, name=None)

Bases: CustomNetwork

Simple Deep Neural Network with custom physical loss function.

Note that the phygnn model requires TensorFlow 2.x

Parameters:

• p_fun (function) – Physics function to guide the neural network loss function. This fun must take (phygnn, y_true, y_predicted, p, **p_kwargs) as arguments with datatypes (PhysicsGuidedNeuralNetwork, tf.Tensor, np.ndarray, np.ndarray). The function must return a tf.Tensor object with a single numeric loss value (output.ndim == 0).

• loss_weights (tuple, optional) – Loss weights for the neural network y_true vs. y_predicted and for the p_fun loss, respectively. For example, loss_weights=(0.0, 1.0) would simplify the phygnn loss function to just the p_fun output.

• n_features (int, optional) – Number of input features. This should match the last dimension of the feature training data.

• n_labels (int, optional) – Number of output labels. This should match the last dimension of the label training data.

• hidden_layers (list, optional) – List of dictionaries of key word arguments for each hidden layer in the NN. Dense linear layers can be input with their activations or separately for more explicit control over the layer ordering. For example, this is a valid input for hidden_layers that will yield 8 hidden layers (10 layers including input+output):

  [{‘units’: 64, ‘activation’: ‘relu’, ‘dropout’: 0.01},

  {‘units’: 64}, {‘batch_normalization’: {‘axis’: -1}}, {‘activation’: ‘relu’}, {‘dropout’: 0.01}, {‘class’: ‘Flatten’}, ]

• input_layer (None | bool | dict) – Input layer. specification. Can be a dictionary similar to hidden_layers specifying a dense / conv / lstm layer. Will default to a keras InputLayer with input shape = n_features. Can be False if the input layer will be included in the hidden_layers input.

• output_layer (None | bool | list | dict) – Output layer specification. Can be a list/dict similar to hidden_layers input specifying a dense layer with activation. For example, for a classfication problem with a single output, output_layer should be [{‘units’: 1}, {‘activation’: ‘sigmoid’}]. This defaults to a single dense layer with no activation (best for regression problems). Can be False if the output layer will be included in the hidden_layers input.

• layers_obj (None | phygnn.utilities.tf_layers.Layers) – Optional initialized Layers object to set as the model layers including pre-set weights. This option will override the hidden_layers, input_layer, and output_layer arguments.

• metric (str, optional) – Loss metric option for the NN loss function (not the physical loss function). Must be a valid key in phygnn.loss_metrics.METRICS or a method in tensorflow.keras.losses that takes (y_true, y_predicted) as arguments.

• optimizer (tensorflow.keras.optimizers | dict | None) – Instantiated tf.keras.optimizers object or a dict optimizer config from tf.keras.optimizers.get_config(). None defaults to Adam.

• learning_rate (float, optional) – Optimizer learning rate. Not used if optimizer input arg is a pre-initialized object or if optimizer input arg is a config dict.

• history (None | pd.DataFrame, optional) – Learning history if continuing a training session.

• kernel_reg_rate (float, optional) – Kernel regularization rate. Increasing this value above zero will add a structural loss term to the loss function that disincentivizes large hidden layer weights and should reduce model complexity. Setting this to 0.0 will disable kernel regularization.

• kernel_reg_power (int, optional) – Kernel regularization power. kernel_reg_power=1 is L1 regularization (lasso regression), and kernel_reg_power=2 is L2 regularization (ridge regression).

• bias_reg_rate (float, optional) – Bias regularization rate. Increasing this value above zero will add a structural loss term to the loss function that disincentivizes large hidden layer biases and should reduce model complexity. Setting this to 0.0 will disable bias regularization.

• bias_reg_power (int, optional) – Bias regularization power. bias_reg_power=1 is L1 regularization (lasso regression), and bias_reg_power=2 is L2 regularization (ridge regression).

• feature_names (list | tuple | None, optional) – Training feature names (strings). Mostly a convenience so that a loaded-from-disk model will have declared feature names, making it easier to feed in features for prediction. This will also get set if phygnn is trained on a DataFrame.

• output_names (list | tuple | None, optional) – Prediction output names (strings). Mostly a convenience so that a loaded-from-disk model will have declared output names, making it easier to understand prediction output. This will also get set if phygnn is trained on a DataFrame.

• name (None | str) – Optional model name for debugging.

Methods

calc_loss(y_true, y_predicted, p, p_kwargs)	Calculate the loss function by comparing y_true to model-predicted y
fit(x, y, p[, n_batch, batch_size, n_epoch, ...])	Fit the neural network to data from x and y.
get_val_split(*args[, shuffle, validation_split])	Get a validation split and remove from from the training data.
load(fpath)	Load a phygnn model that has been saved to a pickle file.
make_batches(*args[, n_batch, batch_size, ...])	Make lists of unique data batches by splitting x and y along the 1st data dimension.
p_fun_dummy(model, y_true, y_predicted, p)	Example dummy function for p loss calculation.
predict(x[, to_numpy, training, training_layers])	Run a prediction on input features.
preflight_data(x, y, p)	Run simple preflight checks on data shapes and data types.
preflight_features(x)	Run preflight checks and data conversions on feature data.
preflight_p_fun(x, y_true, p, p_kwargs)	Run a pre-flight check making sure the p_fun is differentiable.
reset_history()	Erase previous training history without resetting trained weights
run_gradient_descent(x, y_true, p, p_kwargs)	Run gradient descent for one mini-batch of (x, y_true) and adjust NN weights.
save(fpath)	Save phygnn model to pickle file.
seed([s])	Set the random seed for reproducable results.
set_loss_weights(loss_weights)	Set new loss weights

Attributes

bias_reg_term	Get the regularization term for the bias regularization without the regularization rate applied.
bias_weights	Get a list of the NN bias weights (tensors)
history	Model training history DataFrame (None if not yet trained)
kernel_reg_term	Get the regularization term for the kernel regularization without the regularization rate applied.
kernel_weights	Get a list of the NN kernel weights (tensors)
layers	Ordered list of TensorFlow keras layers that make up this model including input and output layers
layers_obj	phygnn layers handler object
model_params	Model parameters, used to save model to disc
version_record	A record of important versions that this model was built with.
weights	Get a list of layer weights and bias terms for gradient calculations.

static p_fun_dummy(model, y_true, y_predicted, p)

Example dummy function for p loss calculation.

This dummy function does not do a real physics calculation, it just shows the required p_fun interface and calculates a normal MAE loss based on y_predicted and y_true.

Parameters:

• model (PhysicsGuidedNeuralNetwork) – Instance of the phygnn model at the current point in training.

• y_true (np.ndarray) – Known y values that were given to the phygnn.fit() method.

• y_predicted (tf.Tensor) – Predicted y values in a >=2D tensor based on x values in the current batch.

• p (np.ndarray) – Supplemental physical feature data that can be used to calculate a y_physical value to compare against y_predicted. The rows in this array have been carried through the batching process alongside y_true and the x-features used to create y_predicted and so can be used 1-to-1 with the rows in y_predicted and y_true.

Returns:

p_loss (tf.Tensor) – A 0D tensor physical loss value.

preflight_data(x, y, p)

Run simple preflight checks on data shapes and data types.

Parameters:

• x (np.ndarray | pd.DataFrame) – Feature data in a >=2D array or DataFrame. If this is a DataFrame, the index is ignored, the columns are used with self.feature_names, and the df is converted into a numpy array for batching and passing to the training algorithm. Generally speaking, the data should always have the number of observations in the first axis and the number of features/channels in the last axis. Spatial and temporal dimensions can be used in intermediate axes.

• y (np.ndarray | pd.DataFrame) – Known output data in a >=2D array or DataFrame. Same dimension rules as x.

• p (np.ndarray | pd.DataFrame) – Supplemental feature data for the physics loss function in >=2D array or DataFrame. Same dimension rules as x.

Returns:

• x (np.ndarray) – Feature data

• y (np.ndarray) – Known output data

• p (np.ndarray) – Supplemental feature data

property history

Model training history DataFrame (None if not yet trained)

Returns:

pandas.DataFrame | None

property kernel_reg_term

Get the regularization term for the kernel regularization without the regularization rate applied.

property bias_reg_term

Get the regularization term for the bias regularization without the regularization rate applied.

property model_params

Model parameters, used to save model to disc

Returns:

dict

preflight_p_fun(x, y_true, p, p_kwargs)

Run a pre-flight check making sure the p_fun is differentiable.

reset_history()

Erase previous training history without resetting trained weights

set_loss_weights(loss_weights)

Set new loss weights

Parameters:

loss_weights (tuple) – Loss weights for the neural network y_true vs y_predicted and for the p_fun loss, respectively. For example, loss_weights=(0.0, 1.0) would simplify the phygnn loss function to just the p_fun output.

property bias_weights

Get a list of the NN bias weights (tensors)

(can be used for bias regularization).

Does not include input layer or dropout layers. Does include the output layer.

Returns:

list

calc_loss(y_true, y_predicted, p, p_kwargs)

Calculate the loss function by comparing y_true to model-predicted y

Parameters:

• y_true (np.ndarray) – Known output data in a >=2D array.

• y_predicted (tf.Tensor) – Model-predicted output data in a >=2D tensor.

• p (np.ndarray) – Supplemental feature data for the physics loss function in >=2D array

• p_kwargs (None | dict) – Optional kwargs for the physical loss function self._p_fun.

Returns:

• loss (tf.tensor) – Sum of the NN loss function comparing the y_predicted against y_true and the physical loss function (self._p_fun) with respective weights applied.

• nn_loss (tf.tensor) – Standard NN training loss comparing y to y_predicted.

• p_loss (tf.tensor) – Physics loss from p_fun.

classmethod get_val_split(*args, shuffle=True, validation_split=0.2)

Get a validation split and remove from from the training data. This applies the split along the 1st data dimension.

Parameters:

• args (np.ndarray) – This is one or more positional arguments that are numpy arrays to be split. They must have the same length.

• shuffle (bool) – Flag to randomly subset the validation data from x and y. shuffle=False will take the first entries in x and y.

• validation_split (float) – Fraction of x and y to put in the validation set.

Returns:

out (list) – List with the same length as the number of positional input arguments. Each list entry is itself a list with two entries. For example, the first entry in the output is of the format: [the training split, and the validation split] and corresponds to the first positional input argument.

property kernel_weights

Get a list of the NN kernel weights (tensors)

(can be used for kernel regularization).

Does not include input layer or dropout layers. Does include the output layer.

Returns:

list

property layers

Ordered list of TensorFlow keras layers that make up this model including input and output layers

Returns:

list

property layers_obj

phygnn layers handler object

Returns:

phygnn.utilities.tf_layers.Layers

classmethod load(fpath)

Load a phygnn model that has been saved to a pickle file.

Parameters:

fpath (str) – File path to .pkl file to load model from.

Returns:

model (PhysicsGuidedNeuralNetwork) – Instantiated phygnn model

static make_batches(*args, n_batch=16, batch_size=None, shuffle=True)

Make lists of unique data batches by splitting x and y along the 1st data dimension.

Parameters:

• args (np.ndarray) – This is one or more positional arguments that are numpy arrays to be batched. They must have the same length.

• n_batch (int | None) – Number of times to update the NN weights per epoch. The training data will be split into this many batches and the NN will train on each batch, update weights, then move onto the next batch.

• batch_size (int | None) – Number of training samples per batch. This input is redundant to n_batch and will not be used if n_batch is not None.

• shuffle (bool) – Flag to randomly subset the validation data from x and y.

Returns:

batches (GeneratorType) – Generator of batches, each iteration of the generator has as many entries as are input in the positional arguments. Each entry in the iteration is an ND array with the same original dimensions as the input just with a subset batch of the 0 axis

predict(x, to_numpy=True, training=False, training_layers=(<class 'keras.src.layers.normalization.batch_normalization.BatchNormalization'>, <class 'keras.src.layers.regularization.dropout.Dropout'>, <class 'keras.src.layers.rnn.lstm.LSTM'>))

Run a prediction on input features.

Parameters:

• x (np.ndarray | pd.DataFrame) – Feature data in a >=2D array or DataFrame. If this is a DataFrame, the index is ignored, the columns are used with self.feature_names, and the df is converted into a numpy array for batching and passing to the training algorithm. Generally speaking, the data should always have the number of observations in the first axis and the number of features/channels in the last axis. Spatial and temporal dimensions can be used in intermediate axes.

• to_numpy (bool) – Flag to convert output from tensor to numpy array

• training (bool) – Flag for predict() used in the training routine. This is used to freeze the BatchNormalization and Dropout layers.

• training_layers (list | tuple) – List of tensorflow.keras.layers classes that training=bool should be passed to. By default this is (BatchNormalization, Dropout, LSTM)

Returns:

y (tf.Tensor | np.ndarray) – Predicted output data.

preflight_features(x)

Run preflight checks and data conversions on feature data.

Parameters:

x (np.ndarray | pd.DataFrame) – Feature data in a >=2D array or DataFrame. If this is a DataFrame, the index is ignored, the columns are used with self.feature_names, and the df is converted into a numpy array for batching and passing to the training algorithm. Generally speaking, the data should always have the number of observations in the first axis and the number of features/channels in the last axis. Spatial and temporal dimensions can be used in intermediate axes.

Returns:

x (np.ndarray) – Feature data in a >=2D array

save(fpath)

Save phygnn model to pickle file.

Parameters:

fpath (str) – File path to .pkl file to save model to.

static seed(s=0)

Set the random seed for reproducable results.

Parameters:

s (int) – Random seed

property version_record

A record of important versions that this model was built with.

Returns:

dict

property weights

Get a list of layer weights and bias terms for gradient calculations.

Returns:

list

run_gradient_descent(x, y_true, p, p_kwargs)

Run gradient descent for one mini-batch of (x, y_true) and adjust NN weights.

fit(x, y, p, n_batch=16, batch_size=None, n_epoch=10, shuffle=True, validation_split=0.2, p_kwargs=None, run_preflight=True, return_diagnostics=False)

Fit the neural network to data from x and y.

Parameters:

• x (np.ndarray | pd.DataFrame) – Feature data in a >=2D array or DataFrame. If this is a DataFrame, the index is ignored, the columns are used with self.feature_names, and the df is converted into a numpy array for batching and passing to the training algorithm. Generally speaking, the data should always have the number of observations in the first axis and the number of features/channels in the last axis. Spatial and temporal dimensions can be used in intermediate axes.

• y (np.ndarray | pd.DataFrame) – Known output data in a >=2D array or DataFrame. Same dimension rules as x.

• p (np.ndarray | pd.DataFrame) – Supplemental feature data for the physics loss function in >=2D array or DataFrame. Same dimension rules as x.

• n_batch (int | None) – Number of times to update the NN weights per epoch (number of mini-batches). The training data will be split into this many mini-batches and the NN will train on each mini-batch, update weights, then move onto the next mini-batch.

• batch_size (int | None) – Number of training samples per batch. This input is redundant to n_batch and will not be used if n_batch is not None.

• n_epoch (int) – Number of times to iterate on the training data.

• shuffle (bool) – Flag to randomly subset the validation data and batch selection from x, y, and p.

• validation_split (float) – Fraction of x, y, and p to use for validation.

• p_kwargs (None | dict) – Optional kwargs for the physical loss function self._p_fun.

• run_preflight (bool) – Flag to run preflight checks.

• return_diagnostics (bool) – Flag to return training diagnostics dictionary.

Returns:

diagnostics (dict) – Namespace of training parameters that can be used for diagnostics.