soogo.model package

Submodules

• soogo.model.base module
  • Surrogate
    • Surrogate.X
    • Surrogate.Y
    • Surrogate.__call__()
    • Surrogate.check_initial_design()
    • Surrogate.dim
    • Surrogate.eval_kernel()
    • Surrogate.iindex
    • Surrogate.min_design_space_size()
    • Surrogate.ntarget
    • Surrogate.ntrain
    • Surrogate.reserve()
    • Surrogate.reset_data()
    • Surrogate.update()
• soogo.model.gp module
  • GaussianProcess
    • GaussianProcess.scaler
    • GaussianProcess.model
    • GaussianProcess.X
    • GaussianProcess.Y
    • GaussianProcess.__call__()
    • GaussianProcess.check_initial_design()
    • GaussianProcess.eval_kernel()
    • GaussianProcess.expected_improvement()
    • GaussianProcess.iindex
    • GaussianProcess.min_design_space_size()
    • GaussianProcess.reserve()
    • GaussianProcess.reset_data()
    • GaussianProcess.update()
• soogo.model.rbf module
  • MedianLpfFilter
    • MedianLpfFilter.__call__()
  • RbfFilter
    • RbfFilter.__call__()
  • RbfModel
    • RbfModel.kernel
    • RbfModel.filter
    • RbfModel.X
    • RbfModel.Y
    • RbfModel.__call__()
    • RbfModel.check_initial_design()
    • RbfModel.eval_kernel()
    • RbfModel.hessp()
    • RbfModel.iindex
    • RbfModel.jac()
    • RbfModel.min_design_space_size()
    • RbfModel.mu_measure()
    • RbfModel.polynomial_tail()
    • RbfModel.polynomial_tail_size()
    • RbfModel.prepare_mu_measure()
    • RbfModel.reserve()
    • RbfModel.reset_data()
    • RbfModel.update()
• soogo.model.rbf_kernel module
  • CubicRadialBasisFunction
    • CubicRadialBasisFunction.__call__()
    • CubicRadialBasisFunction.grad_over_r()
    • CubicRadialBasisFunction.polynomial_tail_order()
  • LinearRadialBasisFunction
    • LinearRadialBasisFunction.__call__()
    • LinearRadialBasisFunction.polynomial_tail_order()
  • RadialBasisFunction
    • RadialBasisFunction.__call__()
    • RadialBasisFunction.grad()
    • RadialBasisFunction.grad_over_r()
    • RadialBasisFunction.hess()
    • RadialBasisFunction.polynomial_tail_order()
  • ThinPlateRadialBasisFunction
    • ThinPlateRadialBasisFunction.__call__()
    • ThinPlateRadialBasisFunction.grad_over_r()
    • ThinPlateRadialBasisFunction.polynomial_tail_order()

Module contents

soogo models

class soogo.model.CubicRadialBasisFunction

Bases: RadialBasisFunction

__call__(r)

Call self as a function.

grad_over_r(r)

Gradient of the radial function.

static polynomial_tail_order() → int

Return the order of the polynomial tail.

Return type:

int

Returns:

Order of the polynomial tail, or -1 if no polynomial tail is present.

class soogo.model.GaussianProcess(scaler=None, **kwargs) → None

Bases: Surrogate

Gaussian Process model.

This model uses default attributes and parameters from GaussianProcessRegressor with the following exceptions:

• kernel: Default is sklearn.gaussian_process.kernels.RBF().

• optimizer: Default is _optimizer().

• normalize_y: Default is True.

• n_restarts_optimizer: Default is 10.

Check other attributes and parameters for GaussianProcessRegressor at https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.GaussianProcessRegressor.html.

Parameters:

scaler – Scaler for the input data. For details, see https://scikit-learn.org/stable/modules/preprocessing.html. (default: None)

scaler

Scaler used to preprocess input data.

model

The underlying GaussianProcessRegressor model instance. This is initialized with the provided parameters and can be accessed for further customization or inspection.

property X: ndarray

Get the training data points.

Returns:

m-by-d matrix with m training points in a d-dimensional space.

property Y: ndarray

Get f(x) for the sampled points.

__call__(x: ndarray, i: int = -1, return_std: bool = False, return_cov: bool = False)

Evaluates the model at one or multiple points.

Parameters:

• x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

• i (int) – Index of the target dimension to evaluate. If -1, evaluate all. (default: -1)

• return_std (bool) – If True, returns the standard deviation of the predictions. (default: False)

• return_cov (bool) – If True, returns the covariance of the predictions. (default: False)

Returns:

• m-by-n matrix with m predictions.

• If return_std is True, the second output is a m-by-n matrix

  with the standard deviations.

• If return_cov is True, the third output is a m-by-m matrix

  with the covariances if n=1, otherwise it is a m-by-m-by-n matrix.

check_initial_design(sample: ndarray) → bool

Check if the sample is able to generate a valid surrogate.

Parameters:

sample (ndarray) – m-by-d matrix with m training points in a d-dimensional space.

Return type:

bool

eval_kernel(x, y=None)

Evaluate the kernel function at a pair (x,y).

The structure of the kernel is the same as the one passed as parameter but with optimized hyperparameters.

Parameters:

• x – First entry in the tuple (x,y).

• y – Second entry in the tuple (x,y). If None, use x. (default: None)

Returns:

Kernel evaluation result.

expected_improvement(x, ybest)

property iindex: tuple[int, ...]

Return iindex, the sequence of integer variable indexes.

min_design_space_size(dim: int) → int

Return the minimum design space size for a given space dimension.

Return type:

int

reserve(n: int, dim: int, ntarget: int = 1) → None

Reserve space for training data.

Parameters:

• n (int) – Number of training points to reserve.

• dim (int) – Dimension of the input space.

• ntarget (int) – Dimension of the target space. (default: 1)

Return type:

None

reset_data() → None

Reset the surrogate model training data.

This method is used to clear the training data of the surrogate model, allowing it to be reused for a new optimization run.

Return type:

None

update(Xnew, ynew) → None

Updates the model with new pairs of data (x,y).

When the default optimizer method, _optimizer(), is used as optimizer, this routine reports different warnings compared to sklearn.gaussian_process.GaussianProcessRegressor.fit(). The latter reports any convergence failure in L-BFGS-B. This implementation reports the last convergence failure in the multiple L-BFGS-B runs only if there all the runs end up failing. The number of optimization runs is n_restarts_optimizer + 1.

Parameters:

• Xnew – m-by-d matrix with m point coordinates in a d-dimensional space.

• ynew – Function values on the sampled points.

Return type:

None

class soogo.model.LinearRadialBasisFunction

Bases: RadialBasisFunction

__call__(r)

Call self as a function.

static polynomial_tail_order() → int

Return the order of the polynomial tail.

Return type:

int

Returns:

Order of the polynomial tail, or -1 if no polynomial tail is present.

class soogo.model.MedianLpfFilter

Bases: RbfFilter

Filter values by replacing large function values by the median of all.

This strategy was proposed by [1] based on results from [2]. Use this strategy to reduce oscillations of the interpolator, especially if the range target function is large. This filter may reduce the quality of the approximation by the surrogate.

References

[1]

Gutmann, HM. A Radial Basis Function Method for Global Optimization. Journal of Global Optimization 19, 201–227 (2001). https://doi.org/10.1023/A:1011255519438

[2]

Björkman, M., Holmström, K. Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions. Optimization and Engineering 1, 373–397 (2000). https://doi.org/10.1023/A:1011584207202

__call__(x) → ndarray

Call self as a function.

Return type:

ndarray

class soogo.model.RbfModel(kernel: ~soogo.model.rbf_kernel.RadialBasisFunction = <soogo.model.rbf_kernel.CubicRadialBasisFunction object>, iindex: tuple[int, ...] = (), filter: ~soogo.model.rbf.RbfFilter | None = None)

Bases: Surrogate

Radial Basis Function model.

\[f(x) = \sum_{i=1}^{m} \beta_i \phi(\|x - x_i\|) + \sum_{i=1}^{n} \beta_{m+i} p_i(x),\]

where:

• \(m\) is the number of sampled points.

• \(x_i\) are the sampled points.

• \(\beta_i\) are the coefficients of the RBF model.

• \(\phi\) is the kernel function.

• \(p_i\) are the basis functions of the polynomial tail.

• \(n\) is the dimension of the polynomial tail.

This implementation focuses on quick successive updates of the model, which is essential for the good performance of active learning processes.

Parameters:

• kernel (RadialBasisFunction) – Kernel function \(\phi\) used in the RBF model. (default: <soogo.model.rbf_kernel.CubicRadialBasisFunction object at 0x7f33de78a570>)

• iindex (tuple[int, ...]) – Indices of integer variables in the feature space. (default: ())

• filter (Optional[RbfFilter]) – Filter to be used in the target (image) space. (default: None)

kernel

Kernel function \(\phi\) used in the RBF model.

filter

Filter to be used in the target (image) space.

property X: ndarray

Get the training data points.

Returns:

m-by-d matrix with m training points in a d-dimensional space.

property Y: ndarray

Get f(x) for the sampled points.

__call__(x: ndarray, i: int = -1, return_dist: bool = False) → ndarray | Tuple[ndarray, ndarray]

Evaluates the model at one or multiple points.

Parameters:

• x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

• i (int) – Index of the target dimension to evaluate. If -1, evaluate all. (default: -1)

• return_dist (bool) – If True, returns the distance matrix between the input points and the training points. (default: False)

Return type:

Union[ndarray, Tuple[ndarray, ndarray]]

Returns:

• Value for the RBF model on each of the input points.

• Matrix D where D[i, j] is the distance between the i-th

  input point and the j-th training point.

check_initial_design(sample: ndarray) → bool

Check if the sample is able to generate a valid surrogate.

Parameters:

sample (ndarray) – m-by-d matrix with m training points in a d-dimensional space.

Return type:

bool

eval_kernel(x, y=None)

Evaluate the kernel function at a pair (x,y).

The structure of the kernel is the same as the one passed as parameter but with optimized hyperparameters.

Parameters:

• x – First entry in the tuple (x,y).

• y – Second entry in the tuple (x,y). If None, use x. (default: None)

Returns:

Kernel evaluation result.

hessp(x: ndarray, v: ndarray) → ndarray

Evaluates the Hessian of the model at x in the direction of v.

\[H(f)(x) v = \sum_{i=1}^{m} \beta_i \left( \phi''(r_i)\frac{(x^Tv)x}{r_i^2} + \frac{\phi'(r_i)}{r_i} \left(v - \frac{(x^Tv)x}{r_i^2}\right) \right) + \sum_{i=1}^{n} \beta_{m+i} H(p_i)(x) v.\]

where \(r_i = \|x - x_i\|\).

Parameters:

• x (ndarray) – Point in a d-dimensional space.

• v (ndarray) – Direction in which the Hessian is evaluated.

Return type:

ndarray

property iindex: tuple[int, ...]

Return iindex, the sequence of integer variable indexes.

jac(x: ndarray) → ndarray

Evaluates the derivative of the model at one point.

\[\nabla f(x) = \sum_{i=1}^{m} \beta_i \frac{\phi'(r_i)}{r_i} x + \sum_{i=1}^{n} \beta_{m+i} \nabla p_i(x).\]

where \(r_i = \|x - x_i\|\).

Parameters:

x (ndarray) – Point in a d-dimensional space.

Return type:

ndarray

min_design_space_size(dim: int) → int

Return the minimum design space size for a given space dimension.

Return type:

int

mu_measure(x: ndarray) → ndarray

Compute the value of abs(mu) for an RBF model.

The mu measure was first defined in [3] with suggestions of usage for global optimization with RBF functions. In [4], the authors detail the strategy to make the evaluations computationally viable.

The current implementation, uses a different strategy than that from Björkman and Holmström (2000), where a single LDLt factorization is used instead of the QR and Cholesky factorizations. The new algorithm’s performs 10 times less operations than the former. Like the former, the new algorithm is also able to use high-intensity linear algebra operations when the routine is called with multiple points \(x\) are evaluated at once.

Note

Before calling this method, the model must be prepared with prepare_mu_measure().

Parameters:

x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

Return type:

ndarray

Returns:

The value of abs(mu) for every point in x.

References

[3]

Gutmann, HM. A Radial Basis Function Method for Global Optimization. Journal of Global Optimization 19, 201–227 (2001). https://doi.org/10.1023/A:1011255519438

[4]

Björkman, M., Holmström, K. Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions. Optimization and Engineering 1, 373–397 (2000). https://doi.org/10.1023/A:1011584207202

polynomial_tail(x: ndarray) → ndarray

Computes the polynomial tail matrix for a given set of points.

Parameters:

x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

Return type:

ndarray

polynomial_tail_size() → int

Get the dimension of the polynomial tail.

Return type:

int

prepare_mu_measure()

Prepare the model for mu measure computation.

This routine computes the LDLt factorization of the matrix A, which is used to compute the mu measure. The factorization is computed only once and can be reused for multiple calls to mu_measure().

reserve(maxeval: int, dim: int, ntarget: int = 1) → None

Reserve space for the RBF model.

This routine avoids successive dynamic memory allocations with successive calls of update(). If the input maxeval is smaller than the current number of sample points, nothing is done.

Parameters:

• maxeval (int) – Maximum number of function evaluations.

• dim (int) – Dimension of the domain space.

• ntarget (int) – Dimension of the target space. (default: 1)

Return type:

None

reset_data() → None

Reset the surrogate model training data.

This method is used to clear the training data of the surrogate model, allowing it to be reused for a new optimization run.

Return type:

None

update(xNew: ndarray, fx) → None

Updates the model with new pairs of data (x,y).

Parameters:

• xNew (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

• fx – Function values on the sampled points.

Return type:

None

class soogo.model.Surrogate

Bases: ABC

Abstract base class for surrogate models.

This class provides a common interface for different types of surrogate models such as Gaussian Processes and Radial Basis Function models. All surrogate models should inherit from this class and implement the abstract methods.

The interface is designed to support optimization algorithms that use surrogate models for approximating expensive objective functions.

abstract property X: ndarray

Get the training input points.

Returns:

m-by-d matrix with m training points in a d-dimensional space.

abstract property Y: ndarray

Get the training output values.

Returns:

m-by-n matrix with m training points and n target dimensions.

abstractmethod __call__(x: ndarray, i: int = -1, **kwargs) → ndarray | Tuple[ndarray, ...]

Evaluate the surrogate model at given points.

Parameters:

• x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

• i (int) – Index of the target dimension to evaluate. If -1, evaluate all. (default: -1)

Return type:

Union[ndarray, Tuple[ndarray, ...]]

Returns:

Model predictions at the input points. Some models may return additional information such as uncertainty estimates.

abstractmethod check_initial_design(sample: ndarray) → bool

Check if the sample is able to generate a valid surrogate.

Parameters:

sample (ndarray) – m-by-d matrix with m training points in a d-dimensional space.

Return type:

bool

Returns:

True if the sample is valid for building the surrogate model.

property dim: int

Get the dimension (d) of the input space.

Returns:

Dimension of the input space.

abstractmethod eval_kernel(x: ndarray, y: ndarray | None = None) → ndarray

Evaluate the kernel function at a pair (x,y).

The structure of the kernel is the same as the one passed as parameter but with optimized hyperparameters.

Parameters:

• x (ndarray) – First entry in the tuple (x,y).

• y (Optional[ndarray]) – Second entry in the tuple (x,y). If None, use x. (default: None)

Return type:

ndarray

Returns:

Kernel evaluation result.

abstract property iindex: tuple[int, ...]

Return the indices of integer variables in the feature space.

Returns:

Tuple with indices of integer variables.

abstractmethod min_design_space_size(dim: int) → int

Return the minimum design space size for a given space dimension.

Parameters:

dim (int) – Dimension of the space.

Return type:

int

Returns:

Minimum number of points needed to build the surrogate model.

property ntarget: int

Get the dimension (n) of the target space.

Returns:

Dimension of the target space.

property ntrain: int

Get the number of training points (m).

Returns:

Number of training points.

abstractmethod reserve(n: int, dim: int, ntarget: int = 1) → None

Reserve space for training data.

Parameters:

• n (int) – Number of training points to reserve.

• dim (int) – Dimension of the input space.

• ntarget (int) – Dimension of the target space. (default: 1)

Return type:

None

abstractmethod reset_data() → None

Reset the surrogate model training data.

This method is used to clear the training data of the surrogate model, allowing it to be reused for a new optimization run.

Return type:

None

abstractmethod update(x: ndarray, y: ndarray) → None

Update the surrogate model with new training data.

Parameters:

• x (ndarray) – m-by-d matrix with m new point coordinates in a d-dimensional space.

• y (ndarray) – m-by-n matrix with m new function values and n target dimensions.

Return type:

None

class soogo.model.ThinPlateRadialBasisFunction(safe_r=2.2250738585072014e-308)

Bases: RadialBasisFunction

__call__(r)

Call self as a function.

grad_over_r(r)

Gradient of the radial function.

static polynomial_tail_order() → int

Return the order of the polynomial tail.

Return type:

int

Returns:

Order of the polynomial tail, or -1 if no polynomial tail is present.