bayeso.bo

class bayeso.bo.BO(arr_range, str_cov='matern52', str_acq='ei', is_normalized=True, is_ard=True, prior_mu=None, str_optimizer_method_gp='Nelder-Mead', str_optimizer_method_bo='L-BFGS-B', str_modelselection_method='ml', debug=False)

Bases: object

It is a Bayesian optimization class.

Parameters:

• arr_range (numpy.ndarray) – a search space. Shape: (d, 2).
• str_cov (str., optional) – the name of covariance function.
• str_acq (str., optional) – the name of acquisition function.
• is_normalized (bool., optional) – flag for normalizing outputs.
• is_ard (bool., optional) – flag for automatic relevance determination.
• prior_mu (NoneType, or function, optional) – None, or prior mean function.
• str_optimizer_method_gp (str., optional) – the name of optimization method for Gaussian process regression.
• str_optimizer_method_bo (str., optional) – the name of optimization method for Bayesian optimization.
• str_modelselection_method (str., optional) – the name of model selection method for Gaussian process regression.
• debug (bool., optional) – flag for printing log messages.

_get_bounds()

It returns list of range tuples, obtained from self.arr_range.

Returns:list of range tuples. Return type:list

_get_initial_grid(int_grid=50)

It returns grids of self.arr_range.

Parameters:int_grid (int., optional) – the number of grids. Returns:grids of self.arr_range. Shape: (int_grid\(^{\text{d}}\), d). Return type:numpy.ndarray Raises:AssertionError

_get_initial_latin(int_samples)

It returns int_samples examples sampled from Latin hypercube.

Parameters:int_samples (int.) – the number of samples. Returns:examples sampled from Latin hypercube. Shape: (int_samples, d). Return type:numpy.ndarray Raises:AssertionError

_get_initial_sobol(int_samples, int_seed=None)

It returns int_samples examples sampled from Sobol sequence.

Parameters:

• int_samples (int.) – the number of samples.
• int_seed (NoneType or int., optional) – None, or random seed.

Returns:

examples sampled from Sobol sequence. Shape: (int_samples, d).

Return type:

numpy.ndarray

Raises:

AssertionError

_get_initial_uniform(int_samples, int_seed=None)

It returns int_samples examples uniformly sampled.

Parameters:

• int_samples (int.) – the number of samples.
• int_seed (NoneType or int., optional) – None, or random seed.

Returns:

random examples. Shape: (int_samples, d).

Return type:

numpy.ndarray

Raises:

AssertionError

_optimize(fun_negative_acquisition, str_initial_method, int_samples)

It optimizes fun_negative_function with self.str_optimizer_method_bo. int_samples examples are determined by str_initial_method, to start acquisition function optimization.

Parameters:

• fun_objective (function) – negative acquisition function.
• str_initial_method (str.) – the name of sampling method.
• int_samples (int.) – the number of samples.

Returns:

tuple of next point to evaluate and all candidates determined by acquisition function optimization. Shape: ((d, ), (int_samples, d)).

Return type:

(numpy.ndarray, numpy.ndarray)

_optimize_objective(fun_acquisition, X_train, Y_train, X_test, cov_X_X, inv_cov_X_X, hyps)

It returns acquisition function values over X_test.

Parameters:

• fun_acquisition (function) – acquisition function.
• X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d).
• Y_train (numpy.ndarray) – outputs. Shape: (n, 1).
• X_test (numpy.ndarray) – inputs. Shape: (l, d) or (l, m, d).
• cov_X_X (numpy.ndarray) – kernel matrix over X_train. Shape: (n, n).
• inv_cov_X_X (numpy.ndarray) – kernel matrix inverse over X_train. Shape: (n, n).
• hyps (dict.) – dictionary of hyperparameters for Gaussian process.

Returns:

acquisition function values over X_test. Shape: (l, ).

Return type:

numpy.ndarray

get_initial(str_initial_method, fun_objective=None, int_samples=100, int_seed=None)

It returns a single example or int_samples examples, sampled by a certian method str_initial_method.

Parameters:

• str_initial_method (str.) – the name of sampling method.
• fun_objective (NoneType or function, optional) – None, or objective function.
• int_samples (int., optional) – the number of samples.
• int_seed (NoneType or int., optional) – None, or random seed.

Returns:

sampled examples. Shape: (1, d) or (int_samples, d).

Return type:

numpy.ndarray

Raises:

AssertionError

optimize(X_train, Y_train, str_initial_method_ao='uniform', int_samples=100, str_mlm_method='regular')

It computes acquired example, candidates of acquired examples, acquisition function values for the candidates, covariance matrix, inverse matrix of the covariance matrix, hyperparameters optimized, and execution times.

Parameters:

• X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d).
• Y_train (numpy.ndarray) – outputs. Shape: (n, 1).
• str_initial_method_ao (str., optional) – the name of initialization method for acquisition function optimization.
• int_samples (int., optional) – the number of samples.
• str_mlm_method (str., optional) – the name of marginal likelihood maximization method for Gaussian process regression.

Returns:

acquired example and dictionary of information. Shape: ((d, ), dict.).

Return type:

(numpy.ndarray, dict.)

Raises:

AssertionError

bayeso.bo._check_hyps_convergence(list_hyps, hyps, str_cov, is_fixed_noise, ratio_threshold=0.05)

It checks convergence of hyperparameters for Gaussian process regression.

Parameters:

• list_hyps (list) – list of historical hyperparameters for Gaussian process regression.
• hyps (dict.) – dictionary of hyperparameters for acquisition function.
• str_cov (str.) – the name of covariance function.
• is_fixed_noise (bool.) – flag for fixing a noise.
• ratio_threshold (float, optional) – ratio of threshold for checking convergence.

Returns:

flag for checking convergence. If converged, it is True.

Return type:

bool.

Raises:

AssertionError

bayeso.bo._check_optimizer_method_bo(str_optimizer_method_bo, num_dim, debug)

It checks the availability of optimization methods. It helps to run Bayesian optimization, even though additional optimization methods are not installed or there exist the conditions some of optimization methods cannot be run.

Parameters:

• str_optimizer_method_bo (str.) – the name of optimization method for Bayesian optimization.
• num_dim (int.) – dimensionality of the problem we solve.
• debug (bool.) – flag for printing log messages.

Returns:

available str_optimizer_method_bo.

Return type:

str.

Raises:

AssertionError

bayeso.bo._choose_fun_acquisition(str_acq, hyps)

It chooses and returns an acquisition function.

Parameters:

• str_acq (str.) – the name of acquisition function.
• hyps (dict.) – dictionary of hyperparameters for acquisition function.

Returns:

acquisition function.

Return type:

function

Raises:

AssertionError

bayeso.bo.get_best_acquisition(arr_initials, fun_objective)

It returns the best example with respect to values of fun_objective. Here, the best acquisition is a minimizer of fun_objective.

Parameters:

• arr_initials (numpy.ndarray) – inputs. Shape: (n, d).
• fun_objective (function) – an objective function.

Returns:

the best example of arr_initials. Shape: (1, d).

Return type:

numpy.ndarray

Raises:

AssertionError

bayeso.bo.get_grids(arr_ranges, int_grids)

It returns grids of given arr_ranges, where each of dimension has int_grids partitions.

Parameters:

• arr_ranges (numpy.ndarray) – ranges. Shape: (d, 2).
• int_grids (int.) – the number of partitions per dimension.

Returns:

grids of given arr_ranges. Shape: (int_grids\(^{\text{d}}\), d).

Return type:

numpy.ndarray

Raises:

AssertionError