blackboxopt.optimize module

Optimization algorithms for blackboxopt.

class blackboxopt.optimize.OptimizeResult(x: ndarray | None = None, fx: float | ndarray | None = None, nit: int = 0, nfev: int = 0, sample: ndarray | None = None, fsample: ndarray | None = None) → None

Bases: object

Optimization result for the global optimizers provided by this package.

fsample: Optional[ndarray] = None

Vector with all n objective values

fx: Union[float, ndarray, None] = None

Best objective function value

init(fun, bounds, mineval: int, maxeval: int, surrogateModel) → None

Initialize nfev and sample and fsample with data about the optimization that is starting.

This routine calls the objective function nfev times.

Parameters:

• fun – The objective function to be minimized.

• bounds (sequence) – List with the limits [x_min,x_max] of each direction x in the space.

• mineval (int) – Minimum number of function evaluations to build the surrogate model.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModel – Surrogate model to be used.

Return type:

None

init_best_values(surrogateModel) → None

Initialize x and fx based on the best values for the surrogate.

Parameters:

surrogateModel – Surrogate model.

Return type:

None

nfev: int = 0

Number of function evaluations taken

nit: int = 0

Number of active learning iterations

sample: Optional[ndarray] = None

n-by-dim matrix with all n samples

x: Optional[ndarray] = None

Best sample point found so far

blackboxopt.optimize.bayesian_optimization(fun, bounds, maxeval: int, x0y0=(), *, surrogateModel=None, acquisitionFunc: MaximizeEI | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables via active learning of a Gaussian Process model.

See [1] for details.

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• x0y0 – Initial guess for the solution and the value of the objective function at the initial guess. (default: ())

• surrogateModel – Gaussian Process surrogate model. The default is GaussianProcess(). On exit, if provided, the surrogate model is updated to represent the one used in the last iteration. (default: None)

• acquisitionFunc (Optional[MaximizeEI]) – Acquisition function to be used. (default: None)

• batchSize (int) – Number of new sample points to be generated per iteration. The default is 1. (default: 1)

• disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

[1]

Che Y, Müller J, Cheng C. Dispersion-enhanced sequential batch sampling for adaptive contour estimation. Qual Reliab Eng Int. 2024; 40: 131–144. https://doi.org/10.1002/qre.3245

blackboxopt.optimize.cptv(fun, bounds, maxeval: int, x0y0=(), *, surrogateModel: RbfModel | None = None, acquisitionFunc: WeightedAcquisition | None = None, improvementTol: float = 0.001, consecutiveQuickFailuresTol: int = 0, useLocalSearch: bool = False, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using the coordinate perturbation and target value strategy.

This is an implementation of the algorithm desribed in [2]. The algorithm uses a sequence of different acquisition functions as follows:

1. CP step: surrogate_optimization() with acquisitionFunc. Ideally,

   this step would use a WeightedAcquisition object with a NormalSampler sampler. The implementation is configured to use the acquisition proposed by Müller (2016) by default.

2. TV step: surrogate_optimization() with a

   TargetValueAcquisition object.

3. Local step (only when useLocalSearch is True): Runs a local

   continuous optimization with the true objective using the best point found so far as initial guess.

The stopping criteria of steps 1 and 2 is related to the number of consecutive attempts that fail to improve the best solution by at least improvementTol. The algorithm alternates between steps 1 and 2 until there is a sequence (CP,TV,CP) where the individual steps do not meet the successful improvement tolerance. In that case, the algorithm switches to step 3. When the local step is finished, the algorithm goes back top step 1.

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• x0y0 – Initial guess for the solution and the value of the objective function at the initial guess. (default: ())

• surrogateModel (Optional[RbfModel]) – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model is updated to represent the one used in the last iteration. (default: None)

• acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used. If None is provided, a WeightedAcquisition is used following what is described by Müller (2016). (default: None)

• improvementTol (float) – Expected improvement in the global optimum per iteration. (default: 0.001)

• consecutiveQuickFailuresTol (int) – Number of times that the CP step or the TV step fails quickly before the algorithm stops. The default is 0, which means the algorithm will stop after maxeval function evaluations. A quick failure is when the acquisition function in the CP or TV step does not find any significant improvement. (default: 0)

• useLocalSearch (bool) – If True, the algorithm will perform a continuous local search when a significant improvement is not found in a sequence of (CP,TV,CP) steps. (default: False)

• disp (bool) – If True, print information about the optimization process. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

[2]

Müller, J. MISO: mixed-integer surrogate optimization framework. Optim Eng 17, 177–203 (2016). https://doi.org/10.1007/s11081-015-9281-2

blackboxopt.optimize.cptvl(*args, **kwargs) → OptimizeResult

Wrapper to cptv. See cptv().

Return type:

OptimizeResult

blackboxopt.optimize.dycors(fun, bounds, maxeval: int, x0y0=(), *, surrogateModel=None, acquisitionFunc: WeightedAcquisition | None = None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None)

DYCORS algorithm for single-objective optimization

Implementation of the DYCORS (DYnamic COordinate search using Response Surface models) algorithm proposed in [3]. That is a wrapper to surrogate_optimization().

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• x0y0 – Initial guess for the solution and the value of the objective function at the initial guess. (default: ())

• surrogateModel – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model is updated to represent the one used in the last iteration. (default: None)

• acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used. If None is provided, the acquisition function is the one used in DYCORS-LMSRBF from Regis and Shoemaker (2012). (default: None)

• batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

• disp (bool) – If True, print information about the optimization process. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Returns:

The optimization result.

References

[3]

Regis, R. G., & Shoemaker, C. A. (2012). Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Engineering Optimization, 45(5), 529–555. https://doi.org/10.1080/0305215X.2012.687731

blackboxopt.optimize.gosac(fun, gfun, bounds, maxeval: int, *, surrogateModels=(<blackboxopt.rbf.RbfModel object>,), disp: bool = False, callback: ~typing.Callable[[~blackboxopt.optimize.OptimizeResult], None] | None = None)

Minimize a scalar function of one or more variables subject to constraints.

The surrogate models are used to approximate the constraints. The objective function is assumed to be cheap to evaluate, while the constraints are assumed to be expensive to evaluate.

This method is based on [4].

Parameters:

• fun – The objective function to be minimized.

• gfun – The constraint function to be minimized. The constraints must be formulated as g(x) <= 0.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModels – Surrogate models to be used. The default is (RbfModel(),). (default: (<blackboxopt.rbf.RbfModel object at 0x7f0543f86d70>,))

• disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Returns:

The optimization result.

References

[4]

Juliane Müller and Joshua D. Woodbury. 2017. GOSAC: global optimization with surrogate approximation of constraints. J. of Global Optimization 69, 1 (September 2017), 117–136. https://doi.org/10.1007/s10898-017-0496-y

blackboxopt.optimize.initialize_moo_surrogate(fun, bounds, mineval: int, maxeval: int, *, surrogateModels=(<blackboxopt.rbf.RbfModel object>, )) → OptimizeResult

Initialize the surrogate model and the output of the optimization.

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• mineval (int) – Minimum number of function evaluations to build the surrogate model.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModels – Surrogate models to be used. The default is (RbfModel(),). (default: (<blackboxopt.rbf.RbfModel object at 0x7f054486db40>,))

Return type:

OptimizeResult

Returns:

The optimization result.

blackboxopt.optimize.initialize_surrogate_constraints(fun, gfun, bounds, mineval: int, maxeval: int, *, surrogateModels=(<blackboxopt.rbf.RbfModel object>, )) → OptimizeResult

Initialize the surrogate models for the constraints.

Parameters:

• fun – The objective function to be minimized.

• gfun – The constraint functions. Each constraint function must return a scalar value. If the constraint function returns a value greater than zero, it is considered a violation of the constraint.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• mineval (int) – Minimum number of function evaluations to build the surrogate model.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModels – Surrogate models to be used. The default is (RbfModel(),). (default: (<blackboxopt.rbf.RbfModel object at 0x7f054486e590>,))

Return type:

OptimizeResult

Returns:

The optimization result.

blackboxopt.optimize.multistart_msrs(fun, bounds, maxeval: int, *, surrogateModel=None, batchSize: int = 1, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using a response surface model approach with restarts.

This implementation generalizes the algorithms Multistart LMSRS from [5]. The general algorithm calls surrogate_optimization() successive times until there are no more function evaluations available. The first time surrogate_optimization() is called with the given, if any, trained surrogate model. Other function calls use an empty surrogate model. This is done to enable truly different starting samples each time.

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModel – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. (default: None)

• batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

• disp (bool) – If True, print information about the optimization process. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

[5]

Rommel G Regis and Christine A Shoemaker. A stochastic radial basis function method for the global optimization of expensive functions. INFORMS Journal on Computing, 19(4):497–509, 2007.

blackboxopt.optimize.socemo(fun, bounds, maxeval: int, *, surrogateModels=(<blackboxopt.rbf.RbfModel object>,), acquisitionFunc: ~blackboxopt.acquisition.WeightedAcquisition | None = None, acquisitionFuncGlobal: ~blackboxopt.acquisition.WeightedAcquisition | None = None, disp: bool = False, callback: ~typing.Callable[[~blackboxopt.optimize.OptimizeResult], None] | None = None)

Minimize a multiobjective function using the surrogate model approach from [6].

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• surrogateModels – Surrogate models to be used. The default is (RbfModel(),). (default: (<blackboxopt.rbf.RbfModel object at 0x7f0543f86440>,))

• acquisitionFunc (Optional[WeightedAcquisition]) – Acquisition function to be used in the CP step. The default is WeightedAcquisition(0). (default: None)

• acquisitionFuncGlobal (Optional[WeightedAcquisition]) – Acquisition function to be used in the global step. The default is WeightedAcquisition(Sampler(0), 0.95). (default: None)

• disp (bool) – If True, print information about the optimization process. The default is False. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. The default is None. (default: None)

Returns:

The optimization result.

References

[6]

Juliane Mueller. SOCEMO: Surrogate Optimization of Computationally Expensive Multiobjective Problems. INFORMS Journal on Computing, 29(4):581-783, 2017. https://doi.org/10.1287/ijoc.2017.0749

blackboxopt.optimize.surrogate_optimization(fun, bounds, maxeval: int, x0y0=(), *, surrogateModel=None, acquisitionFunc: AcquisitionFunction | None = None, batchSize: int = 1, improvementTol: float = 0.001, nSuccTol: int = 3, nFailTol: int = 5, performContinuousSearch: bool = True, termination=None, disp: bool = False, callback: Callable[[OptimizeResult], None] | None = None) → OptimizeResult

Minimize a scalar function of one or more variables using a surrogate model and an acquisition strategy.

This is a more generic implementation of the RBF algorithm described in [7], using multiple ideas from [8] especially in what concerns mixed-integer optimization. Briefly, the implementation works as follows:

1. If a surrogate model or initial sample points are not provided, choose the initial sample using a Symmetric Latin Hypercube design. Evaluate the objective function at the initial sample points.

2. Repeat 3-8 until there are no function evaluations left.

3. Update the surrogate model with the last sample.

4. Acquire a new sample based on the provided acquisition function.

5. Evaluate the objective function at the new sample.

6. Update the optimization solution and best function value if needed.

7. Determine if there is a significant improvement and update counters.

8. Exit after nFailTol successive failures to improve the minimum.

Mind that, when solving mixed-integer optimization, the algorithm may perform a continuous search whenever a significant improvement is found by updating an integer variable. In the continuous search mode, the algorithm executes step 4 only on continuous variables. The continuous search ends when there are no significant improvements for a number of times as in Müller (2016).

Parameters:

• fun – The objective function to be minimized.

• bounds – List with the limits [x_min,x_max] of each direction x in the search space.

• maxeval (int) – Maximum number of function evaluations.

• x0y0 – Initial guess for the solution and the value of the objective function at the initial guess. (default: ())

• surrogateModel – Surrogate model to be used. If None is provided, a RbfModel model with median low-pass filter is used. On exit, if provided, the surrogate model is updated to represent the one used in the last iteration. (default: None)

• acquisitionFunc (Optional[AcquisitionFunction]) – Acquisition function to be used. If None is provided, the TargetValueAcquisition is used. (default: None)

• batchSize (int) – Number of new sample points to be generated per iteration. (default: 1)

• improvementTol (float) – Expected improvement in the global optimum per iteration. (default: 0.001)

• nSuccTol (int) – Number of consecutive successes before updating the acquisition when necessary. A zero value means there is no need to update the acquisition based no the number of successes. (default: 3)

• nFailTol (int) – Number of consecutive failures before updating the acquisition when necessary. A zero value means there is no need to update the acquisition based no the number of failures. (default: 5)

• termination – Termination condition. Possible values: “nFailTol” and None. (default: None)

• performContinuousSearch (bool) – If True, the algorithm will perform a continuous search when a significant improvement is found by updating an integer variable. (default: True)

• disp (bool) – If True, print information about the optimization process. (default: False)

• callback (Optional[Callable[[OptimizeResult], None]]) – If provided, the callback function will be called after each iteration with the current optimization result. (default: None)

Return type:

OptimizeResult

Returns:

The optimization result.

References

[7]

Björkman, M., Holmström, K. Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions. Optimization and Engineering 1, 373–397 (2000). https://doi.org/10.1023/A:1011584207202

[8]

Müller, J. MISO: mixed-integer surrogate optimization framework. Optim Eng 17, 177–203 (2016). https://doi.org/10.1007/s11081-015-9281-2