soogo package

Subpackages

Submodules

Module contents

soogo (Surrogate-based 0-th Order Global Optimization)

class soogo.GaussianProcess(scaler=None, **kwargs) → None

Bases: Surrogate

Gaussian Process model.

This model uses default attributes and parameters from GaussianProcessRegressor with the following exceptions:

  • kernel: Default is sklearn.gaussian_process.kernels.RBF().

  • optimizer: Default is _optimizer().

  • normalize_y: Default is True.

  • n_restarts_optimizer: Default is 10.

Check other attributes and parameters for GaussianProcessRegressor at https://scikit-learn.org/dev/modules/generated/sklearn.gaussian_process.GaussianProcessRegressor.html.

Parameters:

scaler – Scaler for the input data. For details, see https://scikit-learn.org/stable/modules/preprocessing.html. (default: None)

scaler

Scaler used to preprocess input data.

model

The underlying GaussianProcessRegressor model instance. This is initialized with the provided parameters and can be accessed for further customization or inspection.

property X: ndarray

Get the training data points.

Returns:

m-by-d matrix with m training points in a d-dimensional space.

property Y: ndarray

Get f(x) for the sampled points.

__call__(x: ndarray, i: int = -1, return_std: bool = False, return_cov: bool = False)

Evaluates the model at one or multiple points.

Parameters:

  • x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

  • i (int) – Index of the target dimension to evaluate. If -1, evaluate all. (default: -1)

  • return_std (bool) – If True, returns the standard deviation of the predictions. (default: False)

  • return_cov (bool) – If True, returns the covariance of the predictions. (default: False)

Returns:

  • m-by-n matrix with m predictions.

  • If return_std is True, the second output is a m-by-n matrix

    with the standard deviations.

  • If return_cov is True, the third output is a m-by-m matrix

    with the covariances if n=1, otherwise it is a m-by-m-by-n matrix.

check_initial_design(sample: ndarray) → bool

Check if the sample is able to generate a valid surrogate.

Parameters:

sample (ndarray) – m-by-d matrix with m training points in a d-dimensional space.

Return type:

bool

eval_kernel(x, y=None)

Evaluate the kernel function at a pair (x,y).

The structure of the kernel is the same as the one passed as parameter but with optimized hyperparameters.

Parameters:

  • x – First entry in the tuple (x,y).

  • y – Second entry in the tuple (x,y). If None, use x. (default: None)

Returns:

Kernel evaluation result.

expected_improvement(x, ybest)
property iindex: tuple[int, ...]

Return iindex, the sequence of integer variable indexes.

min_design_space_size(dim: int) → int

Return the minimum design space size for a given space dimension.

Return type:

int

reserve(n: int, dim: int, ntarget: int = 1) → None

Reserve space for training data.

Parameters:

  • n (int) – Number of training points to reserve.

  • dim (int) – Dimension of the input space.

  • ntarget (int) – Dimension of the target space. (default: 1)

Return type:

None

reset_data() → None

Reset the surrogate model training data.

This method is used to clear the training data of the surrogate model, allowing it to be reused for a new optimization run.

Return type:

None

update(Xnew, ynew) → None

Updates the model with new pairs of data (x,y).

When the default optimizer method, _optimizer(), is used as optimizer, this routine reports different warnings compared to sklearn.gaussian_process.GaussianProcessRegressor.fit(). The latter reports any convergence failure in L-BFGS-B. This implementation reports the last convergence failure in the multiple L-BFGS-B runs only if there all the runs end up failing. The number of optimization runs is n_restarts_optimizer + 1.

Parameters:

  • Xnew – m-by-d matrix with m point coordinates in a d-dimensional space.

  • ynew – Function values on the sampled points.

Return type:

None

class soogo.MedianLpfFilter

Bases: RbfFilter

Filter values by replacing large function values by the median of all.

This strategy was proposed by [1] based on results from [2]. Use this strategy to reduce oscillations of the interpolator, especially if the range target function is large. This filter may reduce the quality of the approximation by the surrogate.

References

__call__(x) → ndarray

Call self as a function.

Return type:

ndarray

class soogo.OptimizeResult(x: ndarray | None = None, fx: float | ndarray | None = None, nit: int = 0, nfev: int = 0, sample: ndarray | None = None, fsample: ndarray | None = None, nobj: int = 1) → None

Bases: object

Optimization result for the global optimizers provided by this package.

fsample: Optional[ndarray] = None

Vector with all n objective values

fx: Union[float, ndarray, None] = None

Best objective function value

init(fun, bounds, mineval: int, maxeval: int, surrogateModel: Surrogate, ntarget: int = 1) → None

Initialize nfev and sample and fsample with data about the optimization that is starting.

This routine calls the objective function nfev times.

By default, all targets are considered to be used in the objective. If that is not the case, set nobj after calling this function.

Parameters:

  • fun – The objective function to be minimized.

  • bounds (sequence) – List with the limits [x_min,x_max] of each direction x in the space.

  • mineval (int) – Minimum number of function evaluations to build the surrogate model.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Surrogate) – Surrogate model to be used.

  • ntarget (int) – Number of target dimensions. Default is 1. (default: 1)

Return type:

None

init_best_values(surrogateModel: Surrogate | None = None) → None

Initialize x and fx based on the best values obtained so far.

Parameters:

surrogateModel (Optional[Surrogate]) – Surrogate model. (default: None)

Return type:

None

nfev: int = 0

Number of function evaluations taken

nit: int = 0

Number of active learning iterations

nobj: int = 1

Number of objective function targets

sample: Optional[ndarray] = None

n-by-dim matrix with all n samples

x: Optional[ndarray] = None

Best sample point found so far

class soogo.RbfModel(kernel: ~soogo.model.rbf_kernel.RadialBasisFunction = <soogo.model.rbf_kernel.CubicRadialBasisFunction object>, iindex: tuple[int, ...] = (), filter: ~soogo.model.rbf.RbfFilter | None = None)

Bases: Surrogate

Radial Basis Function model.

\[f(x) = \sum_{i=1}^{m} \beta_i \phi(\|x - x_i\|) + \sum_{i=1}^{n} \beta_{m+i} p_i(x),\]

where:

  • \(m\) is the number of sampled points.

  • \(x_i\) are the sampled points.

  • \(\beta_i\) are the coefficients of the RBF model.

  • \(\phi\) is the kernel function.

  • \(p_i\) are the basis functions of the polynomial tail.

  • \(n\) is the dimension of the polynomial tail.

This implementation focuses on quick successive updates of the model, which is essential for the good performance of active learning processes.

Parameters:

  • kernel (RadialBasisFunction) – Kernel function \(\phi\) used in the RBF model. (default: <soogo.model.rbf_kernel.CubicRadialBasisFunction object at 0x7f33de78a570>)

  • iindex (tuple[int, ...]) – Indices of integer variables in the feature space. (default: ())

  • filter (Optional[RbfFilter]) – Filter to be used in the target (image) space. (default: None)

kernel

Kernel function \(\phi\) used in the RBF model.

filter

Filter to be used in the target (image) space.

property X: ndarray

Get the training data points.

Returns:

m-by-d matrix with m training points in a d-dimensional space.

property Y: ndarray

Get f(x) for the sampled points.

__call__(x: ndarray, i: int = -1, return_dist: bool = False) → ndarray | Tuple[ndarray, ndarray]

Evaluates the model at one or multiple points.

Parameters:

  • x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

  • i (int) – Index of the target dimension to evaluate. If -1, evaluate all. (default: -1)

  • return_dist (bool) – If True, returns the distance matrix between the input points and the training points. (default: False)

Return type:

Union[ndarray, Tuple[ndarray, ndarray]]

Returns:

  • Value for the RBF model on each of the input points.

  • Matrix D where D[i, j] is the distance between the i-th

    input point and the j-th training point.

check_initial_design(sample: ndarray) → bool

Check if the sample is able to generate a valid surrogate.

Parameters:

sample (ndarray) – m-by-d matrix with m training points in a d-dimensional space.

Return type:

bool

eval_kernel(x, y=None)

Evaluate the kernel function at a pair (x,y).

The structure of the kernel is the same as the one passed as parameter but with optimized hyperparameters.

Parameters:

  • x – First entry in the tuple (x,y).

  • y – Second entry in the tuple (x,y). If None, use x. (default: None)

Returns:

Kernel evaluation result.

hessp(x: ndarray, v: ndarray) → ndarray

Evaluates the Hessian of the model at x in the direction of v.

\[H(f)(x) v = \sum_{i=1}^{m} \beta_i \left( \phi''(r_i)\frac{(x^Tv)x}{r_i^2} + \frac{\phi'(r_i)}{r_i} \left(v - \frac{(x^Tv)x}{r_i^2}\right) \right) + \sum_{i=1}^{n} \beta_{m+i} H(p_i)(x) v.\]

where \(r_i = \|x - x_i\|\).

Parameters:

  • x (ndarray) – Point in a d-dimensional space.

  • v (ndarray) – Direction in which the Hessian is evaluated.

Return type:

ndarray

property iindex: tuple[int, ...]

Return iindex, the sequence of integer variable indexes.

jac(x: ndarray) → ndarray

Evaluates the derivative of the model at one point.

\[\nabla f(x) = \sum_{i=1}^{m} \beta_i \frac{\phi'(r_i)}{r_i} x + \sum_{i=1}^{n} \beta_{m+i} \nabla p_i(x).\]

where \(r_i = \|x - x_i\|\).

Parameters:

x (ndarray) – Point in a d-dimensional space.

Return type:

ndarray

min_design_space_size(dim: int) → int

Return the minimum design space size for a given space dimension.

Return type:

int

mu_measure(x: ndarray) → ndarray

Compute the value of abs(mu) for an RBF model.

The mu measure was first defined in [3] with suggestions of usage for global optimization with RBF functions. In [4], the authors detail the strategy to make the evaluations computationally viable.

The current implementation, uses a different strategy than that from Björkman and Holmström (2000), where a single LDLt factorization is used instead of the QR and Cholesky factorizations. The new algorithm’s performs 10 times less operations than the former. Like the former, the new algorithm is also able to use high-intensity linear algebra operations when the routine is called with multiple points \(x\) are evaluated at once.

Note

Before calling this method, the model must be prepared with prepare_mu_measure().

Parameters:

x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

Return type:

ndarray

Returns:

The value of abs(mu) for every point in x.

References

polynomial_tail(x: ndarray) → ndarray

Computes the polynomial tail matrix for a given set of points.

Parameters:

x (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

Return type:

ndarray

polynomial_tail_size() → int

Get the dimension of the polynomial tail.

Return type:

int

prepare_mu_measure()

Prepare the model for mu measure computation.

This routine computes the LDLt factorization of the matrix A, which is used to compute the mu measure. The factorization is computed only once and can be reused for multiple calls to mu_measure().

reserve(maxeval: int, dim: int, ntarget: int = 1) → None

Reserve space for the RBF model.

This routine avoids successive dynamic memory allocations with successive calls of update(). If the input maxeval is smaller than the current number of sample points, nothing is done.

Parameters:

  • maxeval (int) – Maximum number of function evaluations.

  • dim (int) – Dimension of the domain space.

  • ntarget (int) – Dimension of the target space. (default: 1)

Return type:

None

reset_data() → None

Reset the surrogate model training data.

This method is used to clear the training data of the surrogate model, allowing it to be reused for a new optimization run.

Return type:

None

update(xNew: ndarray, fx) → None

Updates the model with new pairs of data (x,y).

Parameters:

  • xNew (ndarray) – m-by-d matrix with m point coordinates in a d-dimensional space.

  • fx – Function values on the sampled points.

Return type:

None

soogo.bayesian_optimization(fun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, acquisitionFunc: MaximizeEI | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables via active learning of a Gaussian Process model.

See [5] for details.

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Gaussian Process surrogate model. The default is GaussianProcess(). On exit, if provided, the surrogate model the points used during the optimization. (default: None)

  • acquisitionFunc (Optional[MaximizeEI]) – Acquisition function to be used. (default: None)

  • batchSize (int) – Number of new sample points to be generated per iteration. The default is 1. (default: 1)

  • disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

soogo.cptv(fun, bounds, maxeval: int, *, surrogateModel: RbfModel | None = None, acquisitionFunc: WeightedAcquisition | None = None, improvementTol: float = 0.001, consecutiveQuickFailuresTol: int = 0, useLocalSearch: bool = False, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using the coordinate perturbation and target value strategy.

This is an implementation of the algorithm desribed in [6]. The algorithm uses a sequence of different acquisition functions as follows:

  1. CP step: surrogate_optimization() with acquisitionFunc. Ideally,

    this step would use a WeightedAcquisition object with a NormalSampler sampler. The implementation is configured to use the acquisition proposed by Müller (2016) by default.

  2. TV step: surrogate_optimization() with a

    TargetValueAcquisition object.

  3. Local step (only when useLocalSearch is True): Runs a local

    continuous optimization with the true objective using the best point found so far as initial guess.

The stopping criteria of steps 1 and 2 is related to the number of consecutive attempts that fail to improve the best solution by at least improvementTol. The algorithm alternates between steps 1 and 2 until there is a sequence (CP,TV,CP) where the individual steps do not meet the successful improvement tolerance. In that case, the algorithm switches to step 3. When the local step is finished, the algorithm goes back top step 1.

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[RbfModel]) – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model the points used during the optimization. (default: None)

  • acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used. If None is provided, a WeightedAcquisition is used following what is described by Müller (2016). (default: None)

  • improvementTol (float) – Expected improvement in the global optimum per iteration. (default: 0.001)

  • consecutiveQuickFailuresTol (int) – Number of times that the CP step or the TV step fails quickly before the algorithm stops. The default is 0, which means the algorithm will stop after maxeval function evaluations. A quick failure is when the acquisition function in the CP or TV step does not find any significant improvement. (default: 0)

  • useLocalSearch (bool) – If True, the algorithm will perform a continuous local search when a significant improvement is not found in a sequence of (CP,TV,CP) steps. (default: False)

  • disp (bool) – If True, print information about the optimization process. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

soogo.cptvl(*args, **kwargs) → OptimizeResult

Wrapper to cptv. See cptv().

Return type:

OptimizeResult

soogo.dycors(fun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, acquisitionFunc: WeightedAcquisition | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None)

DYCORS algorithm for single-objective optimization

Implementation of the DYCORS (DYnamic COordinate search using Response Surface models) algorithm proposed in [7]. That is a wrapper to surrogate_optimization().

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model the points used during the optimization. (default: None)

  • acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used. If None is provided, the acquisition function is the one used in DYCORS-LMSRBF from Regis and Shoemaker (2012). (default: None)

  • batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

  • disp (bool) – If True, print information about the optimization process. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Returns:

The optimization result.

References

soogo.gosac(fun, gfun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None)

Minimize a scalar function of one or more variables subject to constraints.

The surrogate models are used to approximate the constraints. The objective function is assumed to be cheap to evaluate, while the constraints are assumed to be expensive to evaluate.

This method is based on [8].

Parameters:

  • fun – The objective function to be minimized.

  • gfun – The constraint function to be minimized. The constraints must be formulated as g(x) <= 0.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Surrogate model to be used for the constraints. If None is provided, a RbfModel model is used. (default: None)

  • disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Returns:

The optimization result.

References

soogo.multistart_msrs(fun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using a response surface model approach with restarts.

This implementation generalizes the algorithms Multistart LMSRS from [9]. The general algorithm calls surrogate_optimization() successive times until there are no more function evaluations available. The first time surrogate_optimization() is called with the given, if any, trained surrogate model. Other function calls use an empty surrogate model. This is done to enable truly different starting samples each time.

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. (default: None)

  • batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

  • disp (bool) – If True, print information about the optimization process. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

soogo.socemo(fun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, acquisitionFunc: WeightedAcquisition | None = None, acquisitionFuncGlobal: WeightedAcquisition | None = None, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None)

Minimize a multiobjective function using the surrogate model approach from [10].

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Multi-target surrogate model to be used. If None is provided, a RbfModel model is used. (default: None)

  • acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used in the CP step. The default is WeightedAcquisition(0). (default: None)

  • acquisitionFuncGlobal (Optional[WeightedAcquisition]) – Acquisition function to be used in the global step. The default is WeightedAcquisition(Sampler(0), 0.95). (default: None)

  • disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Returns:

The optimization result.

References

soogo.surrogate_optimization(fun, bounds, maxeval: int, *, surrogateModel: Surrogate | None = None, acquisitionFunc: AcquisitionFunction | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using a surrogate model and an acquisition strategy.

This is a more generic implementation of the RBF algorithm described in [11], using multiple ideas from [12] especially in what concerns mixed-integer optimization. Briefly, the implementation works as follows:

  1. If a surrogate model or initial sample points are not provided, choose the initial sample using a Symmetric Latin Hypercube design. Evaluate the objective function at the initial sample points.

  2. Repeat 3-8 until there are no function evaluations left.

  3. Update the surrogate model with the last sample.

  4. Acquire a new sample based on the provided acquisition function.

  5. Evaluate the objective function at the new sample.

  6. Update the optimization solution and best function value if needed.

  7. Determine if there is a significant improvement and update counters.

  8. Exit after nFailTol successive failures to improve the minimum.

Mind that, when solving mixed-integer optimization, the algorithm may perform a continuous search whenever a significant improvement is found by updating an integer variable. In the continuous search mode, the algorithm executes step 4 only on continuous variables. The continuous search ends when there are no significant improvements for a number of times as in Müller (2016).

Parameters:

  • fun – The objective function to be minimized.

  • bounds – List with the limits [x_min,x_max] of each direction x in the search space.

  • maxeval (int) – Maximum number of function evaluations.

  • surrogateModel (Optional[Surrogate]) – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model the points used during the optimization. (default: None)

  • acquisitionFunc (Optional[AcquisitionFunction]) – Acquisition function to be used. If None is provided, the TargetValueAcquisition is used. (default: None)

  • batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

  • improvementTol – Expected improvement in the global optimum per iteration.

  • nSuccTol – Number of consecutive successes before updating the acquisition when necessary. A zero value means there is no need to update the acquisition based no the number of successes.

  • nFailTol – Number of consecutive failures before updating the acquisition when necessary. A zero value means there is no need to update the acquisition based no the number of failures.

  • termination – Termination condition. Possible values: “nFailTol” and None.

  • performContinuousSearch – If True, the algorithm will perform a continuous search when a significant improvement is found by updating an integer variable.

  • disp (bool) – If True, print information about the optimization process. (default: False)

  • callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References