bayeso: A Bayesian optimization framework in Python¶

bayeso is a simple, but essential Bayesian optimization package, written in Python. It is developed by machine learning group at POSTECH. This project is licensed under the MIT license.

This documentation describes the details of implementation, getting started guides, some examples with bayeso, and Python API specifications. The code can be found in our GitHub repository.

About bayeso¶

Simple, but essential Bayesian optimization package. It is designed to run advanced Bayesian optimization with implementation-specific and application-specific modifications as well as to run Bayesian optimization in various applications simply. This package contains the codes for Gaussian process regression and Gaussian process-based Bayesian optimization. Some famous benchmark and custom benchmark functions for Bayesian optimization are included in bayeso-benchmarks, which can be used to test the Bayesian optimization strategy. If you are interested in this package, please refer to that repository.

Supported Python Version¶

We test our package in the following versions.

Python 2.7 (It will be excluded due to the maintenance schedule for Python 2.7, but it is currently tested.)
Python 3.6
Python 3.7
Python 3.8

Contributor¶

Jungtaek Kim (POSTECH)

Citation¶

@misc{KimJ2017bayeso,
    author={Kim, Jungtaek and Choi, Seungjin},
    title={{bayeso}: A {Bayesian} optimization framework in {Python}},
    howpublished={\url{http://bayeso.org}},
    year={2017}
}

Contact¶

Jungtaek Kim: jtkim@postech.ac.kr

License¶

MIT License

About Bayesian Optimization¶

Bayesian optimization is a global optimization strategy for black-box and expensive-to-evaluate functions. Generic Bayesian optimization follows these steps:

Build a surrogate function with historical inputs and their observations.
Compute and maximize an acquisition function, defined by the outputs of surrogate function.
Observe the maximizer of acquisition function from a true objective function.
Accumulate the maximizer and its observation.

This project helps us to execute this Bayesian optimization procedure. In particular, Gaussian process regression is used as a surrogate function, and various acquisition functions such as probability improvement, expected improvement, and Gaussian process upper confidence bound are included in this project.

Installing bayeso¶

We recommend it should be installed in virtualenv. You can choose one of three installation options.

Installing from PyPI¶

It is for user installation. To install the released version from PyPI repository, command it.

$ pip install bayeso

Compiling from Source¶

It is for developer installation. To install bayeso from source code, command

$ pip install .

in the bayeso root.

Compiling from Source (Editable)¶

It is for editable development mode. To use editable development mode, command

$ pip install -r requirements.txt
$ python setup.py develop

in the bayeso root.

Uninstalling¶

If you would like to uninstall bayeso, command it.

$ pip uninstall bayeso

Required Packages¶

Mandatory pacakges are inlcuded in requirements.txt. The following requirements files include the package list, the purpose of which is described as follows.

requirements-optional.txt: It is an optional package list, but it needs to be installed to execute some features of bayeso.
requirements-dev.txt: It is for developing the bayeso package.
requirements-examples.txt: It needs to be installed to execute the examples included in the bayeso repository.

Building Gaussian Process Regression¶

This example is for building Gaussian process regression models. First of all, import the packages we need and bayeso.

import numpy as np
import os

from bayeso import covariance
from bayeso.gp import gp
from bayeso.utils import utils_covariance
from bayeso.utils import utils_plotting

Declare some parameters to control this example.

is_tex = False
num_test = 200
str_cov = 'matern52'

Make a simple synthetic dataset, which produces with cosine functions.

X_train = np.array([
    [-3.0],
    [-2.0],
    [-1.0],
    [2.0],
    [1.2],
    [1.1],
])
Y_train = np.cos(X_train) + 10.0
X_test = np.linspace(-3, 3, num_test)
X_test = X_test.reshape((num_test, 1))
Y_test_truth = np.cos(X_test) + 10.0

Sample functions from a prior distribution, which is zero mean.

mu = np.zeros(num_test)
hyps = utils_covariance.get_hyps(str_cov, 1)
Sigma = covariance.cov_main(str_cov, X_test, X_test, hyps, True)

Ys = gp.sample_functions(mu, Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

Build a Gaussian process regression model with fixed hyperparameters. Then, plot the result.

hyps = utils_covariance.get_hyps(str_cov, 1)
mu, sigma, Sigma = gp.predict_test(X_train, Y_train, X_test, hyps, str_cov=str_cov)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

Build a Gaussian process regression model with the hyperparameters optimized by marginal likelihood maximization, and plot the result.

mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

Declare some functions that would be employed as prior functions.

def cosine(X):
    return np.cos(X)

def linear_down(X):
    list_up = []
    for elem_X in X:
        list_up.append([-0.5 * np.sum(elem_X)])
    return np.array(list_up)

def linear_up(X):
    list_up = []
    for elem_X in X:
        list_up.append([0.5 * np.sum(elem_X)])
    return np.array(list_up)

Make an another synthetic dataset using a cosine function.

X_train = np.array([
    [-3.0],
    [-2.0],
    [-1.0],
])
Y_train = np.cos(X_train) + 2.0
X_test = np.linspace(-3, 6, num_test)
X_test = X_test.reshape((num_test, 1))
Y_test_truth = np.cos(X_test) + 2.0

Build Gaussian process regression models with the prior functions we declare above and the hyperparameters optimized by marginal likelihood maximization, and plot the result.

prior_mu = cosine
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

prior_mu = linear_down
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

prior_mu = linear_up
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

Full code:

import numpy as np
import os

from bayeso import covariance
from bayeso.gp import gp
from bayeso.utils import utils_covariance
from bayeso.utils import utils_plotting

is_tex = False
num_test = 200
str_cov = 'matern52'

X_train = np.array([
    [-3.0],
    [-2.0],
    [-1.0],
    [2.0],
    [1.2],
    [1.1],
])
Y_train = np.cos(X_train) + 10.0
X_test = np.linspace(-3, 3, num_test)
X_test = X_test.reshape((num_test, 1))
Y_test_truth = np.cos(X_test) + 10.0

mu = np.zeros(num_test)
hyps = utils_covariance.get_hyps(str_cov, 1)
Sigma = covariance.cov_main(str_cov, X_test, X_test, hyps, True)

Ys = gp.sample_functions(mu, Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

hyps = utils_covariance.get_hyps(str_cov, 1)
mu, sigma, Sigma = gp.predict_test(X_train, Y_train, X_test, hyps, str_cov=str_cov)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

def cosine(X):
    return np.cos(X)

def linear_down(X):
    list_up = []
    for elem_X in X:
        list_up.append([-0.5 * np.sum(elem_X)])
    return np.array(list_up)

def linear_up(X):
    list_up = []
    for elem_X in X:
        list_up.append([0.5 * np.sum(elem_X)])
    return np.array(list_up)

X_train = np.array([
    [-3.0],
    [-2.0],
    [-1.0],
])
Y_train = np.cos(X_train) + 2.0
X_test = np.linspace(-3, 6, num_test)
X_test = X_test.reshape((num_test, 1))
Y_test_truth = np.cos(X_test) + 2.0

prior_mu = cosine
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

prior_mu = linear_down
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

prior_mu = linear_up
mu, sigma, Sigma = gp.predict_optimized(X_train, Y_train, X_test, str_cov=str_cov, prior_mu=prior_mu)
utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=Y_test_truth, is_tex=is_tex,
                       str_x_axis='$x$', str_y_axis='$y$')

Ys = gp.sample_functions(mu.flatten(), Sigma, num_samples=5)
utils_plotting.plot_gp_sampled(X_test, Ys, is_tex=is_tex,
                               str_x_axis='$x$', str_y_axis='$y$')

Optimizing Sampled Function via Thompson Sampling¶

This example is to optimize a function sampled from a Gaussian process prior via Thompson sampling. First of all, import the packages we need and bayeso.

import numpy as np

from bayeso import covariance
from bayeso.gp import gp
from bayeso.utils import utils_covariance
from bayeso.utils import utils_plotting

Declare some parameters to control this example, including zero-mean prior, and compute a covariance matrix.

num_points = 1000
str_cov = 'se'
int_init = 1
int_iter = 50
int_ts = 10

list_Y_min = []

X = np.expand_dims(np.linspace(-5, 5, num_points), axis=1)
mu = np.zeros(num_points)
hyps = utils_covariance.get_hyps(str_cov, 1)
Sigma = covariance.cov_main(str_cov, X, X, hyps, True)

Optimize a function sampled from a Gaussian process prior. At each iteration, we sample a query point that outputs the mininum value of the function sampled from a Gaussian process posterior.

for ind_ts in range(0, int_ts):
    print('TS:', ind_ts + 1, 'round')
    Y = gp.sample_functions(mu, Sigma, num_samples=1)[0]

    ind_init = np.argmin(Y)
    bx_min = X[ind_init]
    y_min = Y[ind_init]

    ind_random = np.random.choice(num_points)

    X_ = np.expand_dims(X[ind_random], axis=0)
    Y_ = np.expand_dims(np.expand_dims(Y[ind_random], axis=0), axis=1)

    for ind_iter in range(0, int_iter):
        print(ind_iter + 1, 'iteration')

        mu_, sigma_, Sigma_ = gp.predict_optimized(X_, Y_, X, str_cov=str_cov)
        ind_ = np.argmin(gp.sample_functions(np.squeeze(mu_, axis=1), Sigma_, num_samples=1)[0])

        X_ = np.concatenate([X_, [X[ind_]]], axis=0)
        Y_ = np.concatenate([Y_, [[Y[ind_]]]], axis=0)

    list_Y_min.append(Y_ - y_min)

Ys = np.array(list_Y_min)
Ys = np.squeeze(Ys, axis=2)
print(Ys.shape)

Plot the result obtained from the code block above.

utils_plotting.plot_minimum(np.array([Ys]), ['TS'], 1, True,
                            is_tex=True, range_shade=1.0,
                            str_x_axis=r'\textrm{Iteration}',
                            str_y_axis=r'\textrm{Minimum regret}')

Full code:

import numpy as np

from bayeso import covariance
from bayeso.gp import gp
from bayeso.utils import utils_covariance
from bayeso.utils import utils_plotting

num_points = 1000
str_cov = 'se'
int_init = 1
int_iter = 50
int_ts = 10

list_Y_min = []

X = np.expand_dims(np.linspace(-5, 5, num_points), axis=1)
mu = np.zeros(num_points)
hyps = utils_covariance.get_hyps(str_cov, 1)
Sigma = covariance.cov_main(str_cov, X, X, hyps, True)

for ind_ts in range(0, int_ts):
    print('TS:', ind_ts + 1, 'round')
    Y = gp.sample_functions(mu, Sigma, num_samples=1)[0]

    ind_init = np.argmin(Y)
    bx_min = X[ind_init]
    y_min = Y[ind_init]

    ind_random = np.random.choice(num_points)

    X_ = np.expand_dims(X[ind_random], axis=0)
    Y_ = np.expand_dims(np.expand_dims(Y[ind_random], axis=0), axis=1)

    for ind_iter in range(0, int_iter):
        print(ind_iter + 1, 'iteration')

        mu_, sigma_, Sigma_ = gp.predict_optimized(X_, Y_, X, str_cov=str_cov)
        ind_ = np.argmin(gp.sample_functions(np.squeeze(mu_, axis=1), Sigma_, num_samples=1)[0])

        X_ = np.concatenate([X_, [X[ind_]]], axis=0)
        Y_ = np.concatenate([Y_, [[Y[ind_]]]], axis=0)

    list_Y_min.append(Y_ - y_min)

Ys = np.array(list_Y_min)
Ys = np.squeeze(Ys, axis=2)
print(Ys.shape)

utils_plotting.plot_minimum(np.array([Ys]), ['TS'], 1, True,
                            is_tex=True, range_shade=1.0,
                            str_x_axis=r'\textrm{Iteration}',
                            str_y_axis=r'\textrm{Minimum regret}')

Optimizing Branin Function¶

This example is for optimizing Branin function. It needs to install bayeso-benchmarks, which is included in requirements-optional.txt. First, import some packages we need.

import numpy as np
import os

from bayeso import bo
from benchmarks.two_dim_branin import Branin
from bayeso.utils import utils_bo
from bayeso.utils import utils_plotting

Then, declare Branin function we will optimize and a search space for the function.

obj_fun = Branin()
bounds = obj_fun.get_bounds()

def fun_target(X):
    return obj_fun.output(X)

We optimize the objective function with 10 Bayesian optimization rounds and 50 iterations per round with 3 initial random evaluations.

str_fun = 'branin'

int_bo = 10
int_iter = 50
int_init = 3

With BO class in bayeso.bo, optimize the objective function.

model_bo = bo.BO(bounds, debug=False)
list_Y = []
list_time = []

for ind_bo in range(0, int_bo):
    print('BO Iteration', ind_bo + 1)
    X_final, Y_final, time_final, _, _ = utils_bo.optimize_many_with_random_init(
        model_bo, fun_target, int_init, int_iter,
        str_initial_method_bo='uniform', str_initial_method_ao='uniform', int_samples_ao=100,
        int_seed=42 * ind_bo
    )
    list_Y.append(Y_final)
    list_time.append(time_final)

arr_Y = np.array(list_Y)
arr_time = np.array(list_time)

arr_Y = np.expand_dims(np.squeeze(arr_Y), axis=0)
arr_time = np.expand_dims(arr_time, axis=0)

Plot the results in terms of the number of iterations and time.

utils_plotting.plot_minimum(arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Iteration}',
    str_y_axis=r'\textrm{Mininum function value}')
utils_plotting.plot_minimum_time(arr_time, arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Time (sec.)}',
    str_y_axis=r'\textrm{Mininum function value}')

Full code:

import numpy as np
import os

from bayeso import bo
from benchmarks.two_dim_branin import Branin
from bayeso.utils import utils_bo
from bayeso.utils import utils_plotting

obj_fun = Branin()
bounds = obj_fun.get_bounds()

def fun_target(X):
    return obj_fun.output(X)

str_fun = 'branin'

int_bo = 10
int_iter = 50
int_init = 3

model_bo = bo.BO(bounds, debug=False)
list_Y = []
list_time = []

for ind_bo in range(0, int_bo):
    print('BO Iteration', ind_bo + 1)
    X_final, Y_final, time_final, _, _ = utils_bo.optimize_many_with_random_init(
        model_bo, fun_target, int_init, int_iter,
        str_initial_method_bo='uniform', str_initial_method_ao='uniform', int_samples_ao=100,
        int_seed=42 * ind_bo
    )
    list_Y.append(Y_final)
    list_time.append(time_final)

arr_Y = np.array(list_Y)
arr_time = np.array(list_time)

arr_Y = np.expand_dims(np.squeeze(arr_Y), axis=0)
arr_time = np.expand_dims(arr_time, axis=0)

utils_plotting.plot_minimum(arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Iteration}',
    str_y_axis=r'\textrm{Mininum function value}')
utils_plotting.plot_minimum_time(arr_time, arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Time (sec.)}',
    str_y_axis=r'\textrm{Mininum function value}')

Constructing xgboost Classifier with Hyperparameter Optimization¶

This example is for optimizing hyperparameters for xgboost classifier. In this example, we optimize max_depth and n_estimators for xgboost.XGBClassifier. It needs to install xgboost, which is included in requirements-examples.txt. First, import some packages we need.

import numpy as np
import xgboost as xgb
import sklearn.datasets
import sklearn.metrics
import sklearn.model_selection

from bayeso import bo
from bayeso.utils import utils_bo
from bayeso.utils import utils_plotting

Get handwritten digits dataset, which contains digit images of 0 to 9, and split the dataset to training and test datasets.

digits = sklearn.datasets.load_digits()
data_digits = digits.images
data_digits = np.reshape(data_digits,
    (data_digits.shape[0], data_digits.shape[1] * data_digits.shape[2]))
labels_digits = digits.target

data_train, data_test, labels_train, labels_test = sklearn.model_selection.train_test_split(
    data_digits, labels_digits, test_size=0.3, stratify=labels_digits)

Declare an objective function we would like to optimize. This function trains xgboost.XGBClassifier with the training dataset and given hyerparameter vector bx and returns (1 - accuracy), which computed by the test dataset.

def fun_target(bx):
    model_xgb = xgb.XGBClassifier(
        max_depth=int(bx[0]),
        n_estimators=int(bx[1])
    )
    model_xgb.fit(data_train, labels_train)
    preds_test = model_xgb.predict(data_test)
    return 1.0 - sklearn.metrics.accuracy_score(labels_test, preds_test)

We optimize the objective function with our bayeso.bo.BO for 50 iterations. 5 initial points would be given and 10 rounds would be run.

str_fun = 'xgboost'

# (max_depth, n_estimators)
bounds = np.array([[1, 10], [100, 500]])
int_bo = 10
int_iter = 50
int_init = 5

Optimze the objective function, after declaring the bayeso.bo.BO object.

model_bo = bo.BO(bounds, debug=False)

list_Y = []
list_time = []

for ind_bo in range(0, int_bo):
    print('BO Iteration:', ind_bo + 1)
    X_final, Y_final, time_final, _, _ = utils_bo.optimize_many_with_random_init(
        model_bo, fun_target, int_init, int_iter,
        str_initial_method_bo='uniform', str_initial_method_ao='uniform',
        int_samples_ao=100, int_seed=42 * ind_bo)
    list_Y.append(Y_final)
    list_time.append(time_final)

arr_Y = np.array(list_Y)
arr_time = np.array(list_time)

arr_Y = np.expand_dims(np.squeeze(arr_Y), axis=0)
arr_time = np.expand_dims(arr_time, axis=0)

Plot the results in terms of the number of iterations and time.

utils_plotting.plot_minimum(arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Iteration}',
    str_y_axis=r'$1 - $\textrm{Accuracy}')
utils_plotting.plot_minimum_time(arr_time, arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Time (sec.)}',
    str_y_axis=r'$1 - $\textrm{Accuracy}')

Full code:

import numpy as np
import xgboost as xgb
import sklearn.datasets
import sklearn.metrics
import sklearn.model_selection

from bayeso import bo
from bayeso.utils import utils_bo
from bayeso.utils import utils_plotting

digits = sklearn.datasets.load_digits()
data_digits = digits.images
data_digits = np.reshape(data_digits,
    (data_digits.shape[0], data_digits.shape[1] * data_digits.shape[2]))
labels_digits = digits.target

data_train, data_test, labels_train, labels_test = sklearn.model_selection.train_test_split(
    data_digits, labels_digits, test_size=0.3, stratify=labels_digits)

def fun_target(bx):
    model_xgb = xgb.XGBClassifier(
        max_depth=int(bx[0]),
        n_estimators=int(bx[1])
    )
    model_xgb.fit(data_train, labels_train)
    preds_test = model_xgb.predict(data_test)
    return 1.0 - sklearn.metrics.accuracy_score(labels_test, preds_test)

str_fun = 'xgboost'

# (max_depth, n_estimators)
bounds = np.array([[1, 10], [100, 500]])
int_bo = 10
int_iter = 50
int_init = 5

model_bo = bo.BO(bounds, debug=False)

list_Y = []
list_time = []

for ind_bo in range(0, int_bo):
    print('BO Iteration:', ind_bo + 1)
    X_final, Y_final, time_final, _, _ = utils_bo.optimize_many_with_random_init(
        model_bo, fun_target, int_init, int_iter,
        str_initial_method_bo='uniform', str_initial_method_ao='uniform',
        int_samples_ao=100, int_seed=42 * ind_bo)
    list_Y.append(Y_final)
    list_time.append(time_final)

arr_Y = np.array(list_Y)
arr_time = np.array(list_time)

arr_Y = np.expand_dims(np.squeeze(arr_Y), axis=0)
arr_time = np.expand_dims(arr_time, axis=0)

utils_plotting.plot_minimum(arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Iteration}',
    str_y_axis=r'$1 - $\textrm{Accuracy}')
utils_plotting.plot_minimum_time(arr_time, arr_Y, [str_fun], int_init, True,
    is_tex=True,
    str_x_axis=r'\textrm{Time (sec.)}',
    str_y_axis=r'$1 - $\textrm{Accuracy}')

bayeso¶

bayeso.acquisition¶

bayeso.acquisition.aei(pred_mean, pred_std, Y_train, noise, jitter=1e-05)¶

It is an augmented expected improvement criterion.

Parameters:	pred_mean (numpy.ndarray) – posterior predictive mean function over X_test. Shape: (l, ). pred_std (numpy.ndarray) – posterior predictive standard deviation function over X_test. Shape: (l, ). Y_train (numpy.ndarray) – outputs of X_train. Shape: (n, 1). noise (float) – noise for augmenting exploration. jitter (float, optional) – jitter for pred_std.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.acquisition.ei(pred_mean, pred_std, Y_train, jitter=1e-05)¶

It is an expected improvement criterion.

Parameters:	pred_mean (numpy.ndarray) – posterior predictive mean function over X_test. Shape: (l, ). pred_std (numpy.ndarray) – posterior predictive standard deviation function over X_test. Shape: (l, ). Y_train (numpy.ndarray) – outputs of X_train. Shape: (n, 1). jitter (float, optional) – jitter for pred_std.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.acquisition.pi(pred_mean, pred_std, Y_train, jitter=1e-05)¶

It is a probability improvement criterion.

Parameters:	pred_mean (numpy.ndarray) – posterior predictive mean function over X_test. Shape: (l, ). pred_std (numpy.ndarray) – posterior predictive standard deviation function over X_test. Shape: (l, ). Y_train (numpy.ndarray) – outputs of X_train. Shape: (n, 1). jitter (float, optional) – jitter for pred_std.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.acquisition.pure_exploit(pred_mean, pred_std=None, Y_train=None)¶

It is a pure exploitation criterion.

Parameters:	pred_mean (numpy.ndarray) – posterior predictive mean function over X_test. Shape: (l, ). pred_std (numpy.ndarray, optional) – posterior predictive standard deviation function over X_test. Shape: (l, ). It can be given, but it is ignored when it works. Y_train (numpy.ndarray, optional) – outputs of X_train. Shape: (n, 1). It can be given, but it is ignored when it works.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.acquisition.pure_explore(pred_std, pred_mean=None, Y_train=None)¶

It is a pure exploration criterion.

Parameters:	pred_std (numpy.ndarray) – posterior predictive standard deviation function over X_test. Shape: (l, ). pred_mean (numpy.ndarray, optional) – posterior predictive mean function over X_test. Shape: (l, ). It can be given, but it is ignored when it works. Y_train (numpy.ndarray, optional) – outputs of X_train. Shape: (n, 1). It can be given, but it is ignored when it works.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.acquisition.ucb(pred_mean, pred_std, Y_train=None, kappa=2.0, is_increased=True)¶

It is a Gaussian process upper confidence bound criterion.

Parameters:	pred_mean (numpy.ndarray) – posterior predictive mean function over X_test. Shape: (l, ). pred_std (numpy.ndarray) – posterior predictive standard deviation function over X_test. Shape: (l, ). Y_train (numpy.ndarray, optional) – outputs of X_train. Shape: (n, 1). kappa (float, optional) – trade-off hyperparameter between exploration and exploitation. is_increased (bool., optional) – flag for increasing a kappa value as Y_train grows. If Y_train is None, it is ignored, which means kappa is fixed.
Returns:	acquisition function values. Shape: (l, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.bo¶

class bayeso.bo.BO(arr_range, str_cov='matern52', str_acq='ei', is_normalized=True, is_ard=True, prior_mu=None, str_optimizer_method_gp='Nelder-Mead', str_optimizer_method_bo='L-BFGS-B', str_modelselection_method='ml', debug=False)¶

Bases: object

It is a Bayesian optimization class.

Parameters:

arr_range (numpy.ndarray) – a search space. Shape: (d, 2).
str_cov (str., optional) – the name of covariance function.
str_acq (str., optional) – the name of acquisition function.
is_normalized (bool., optional) – flag for normalizing outputs.
is_ard (bool., optional) – flag for automatic relevance determination.
prior_mu (NoneType, or function, optional) – None, or prior mean function.
str_optimizer_method_gp (str., optional) – the name of optimization method for Gaussian process regression.
str_optimizer_method_bo (str., optional) – the name of optimization method for Bayesian optimization.
str_modelselection_method (str., optional) – the name of model selection method for Gaussian process regression.
debug (bool., optional) – flag for printing log messages.

_get_bounds()¶

It returns list of range tuples, obtained from self.arr_range.

Returns:	list of range tuples.
Return type:	list

_get_initial_grid(int_grid=50)¶

It returns grids of self.arr_range.

Parameters:	int_grid (int., optional) – the number of grids.
Returns:	grids of self.arr_range. Shape: (int_grid\(^{\text{d}}\), d).
Return type:	numpy.ndarray
Raises:	AssertionError

_get_initial_latin(int_samples)¶

It returns int_samples examples sampled from Latin hypercube.

Parameters:	int_samples (int.) – the number of samples.
Returns:	examples sampled from Latin hypercube. Shape: (int_samples, d).
Return type:	numpy.ndarray
Raises:	AssertionError

_get_initial_sobol(int_samples, int_seed=None)¶

It returns int_samples examples sampled from Sobol sequence.

Parameters:	int_samples (int.) – the number of samples. int_seed (NoneType or int., optional) – None, or random seed.
Returns:	examples sampled from Sobol sequence. Shape: (int_samples, d).
Return type:	numpy.ndarray
Raises:	AssertionError

_get_initial_uniform(int_samples, int_seed=None)¶

It returns int_samples examples uniformly sampled.

Parameters:	int_samples (int.) – the number of samples. int_seed (NoneType or int., optional) – None, or random seed.
Returns:	random examples. Shape: (int_samples, d).
Return type:	numpy.ndarray
Raises:	AssertionError

_optimize(fun_negative_acquisition, str_initial_method, int_samples)¶

It optimizes fun_negative_function with self.str_optimizer_method_bo. int_samples examples are determined by str_initial_method, to start acquisition function optimization.

Parameters:	fun_objective (function) – negative acquisition function. str_initial_method (str.) – the name of sampling method. int_samples (int.) – the number of samples.
Returns:	tuple of next point to evaluate and all candidates determined by acquisition function optimization. Shape: ((d, ), (int_samples, d)).
Return type:	(numpy.ndarray, numpy.ndarray)

_optimize_objective(fun_acquisition, X_train, Y_train, X_test, cov_X_X, inv_cov_X_X, hyps)¶

It returns acquisition function values over X_test.

Parameters:	fun_acquisition (function) – acquisition function. X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). X_test (numpy.ndarray) – inputs. Shape: (l, d) or (l, m, d). cov_X_X (numpy.ndarray) – kernel matrix over X_train. Shape: (n, n). inv_cov_X_X (numpy.ndarray) – kernel matrix inverse over X_train. Shape: (n, n). hyps (dict.) – dictionary of hyperparameters for Gaussian process.
Returns:	acquisition function values over X_test. Shape: (l, ).
Return type:	numpy.ndarray

get_initial(str_initial_method, fun_objective=None, int_samples=100, int_seed=None)¶

It returns a single example or int_samples examples, sampled by a certian method str_initial_method.

Parameters:	str_initial_method (str.) – the name of sampling method. fun_objective (NoneType or function, optional) – None, or objective function. int_samples (int., optional) – the number of samples. int_seed (NoneType or int., optional) – None, or random seed.
Returns:	sampled examples. Shape: (1, d) or (int_samples, d).
Return type:	numpy.ndarray
Raises:	AssertionError

optimize(X_train, Y_train, str_initial_method_ao='uniform', int_samples=100, str_mlm_method='regular')¶

It computes acquired example, candidates of acquired examples, acquisition function values for the candidates, covariance matrix, inverse matrix of the covariance matrix, hyperparameters optimized, and execution times.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). str_initial_method_ao (str., optional) – the name of initialization method for acquisition function optimization. int_samples (int., optional) – the number of samples. str_mlm_method (str., optional) – the name of marginal likelihood maximization method for Gaussian process regression.
Returns:	acquired example and dictionary of information. Shape: ((d, ), dict.).
Return type:	(numpy.ndarray, dict.)
Raises:	AssertionError

bayeso.bo._check_hyps_convergence(list_hyps, hyps, str_cov, is_fixed_noise, ratio_threshold=0.05)¶

It checks convergence of hyperparameters for Gaussian process regression.

Parameters:	list_hyps (list) – list of historical hyperparameters for Gaussian process regression. hyps (dict.) – dictionary of hyperparameters for acquisition function. str_cov (str.) – the name of covariance function. is_fixed_noise (bool.) – flag for fixing a noise. ratio_threshold (float, optional) – ratio of threshold for checking convergence.
Returns:	flag for checking convergence. If converged, it is True.
Return type:	bool.
Raises:	AssertionError

bayeso.bo._check_optimizer_method_bo(str_optimizer_method_bo, num_dim, debug)¶

It checks the availability of optimization methods. It helps to run Bayesian optimization, even though additional optimization methods are not installed or there exist the conditions some of optimization methods cannot be run.

Parameters:	str_optimizer_method_bo (str.) – the name of optimization method for Bayesian optimization. num_dim (int.) – dimensionality of the problem we solve. debug (bool.) – flag for printing log messages.
Returns:	available str_optimizer_method_bo.
Return type:	str.
Raises:	AssertionError

bayeso.bo._choose_fun_acquisition(str_acq, hyps)¶

It chooses and returns an acquisition function.

Parameters:	str_acq (str.) – the name of acquisition function. hyps (dict.) – dictionary of hyperparameters for acquisition function.
Returns:	acquisition function.
Return type:	function
Raises:	AssertionError

bayeso.bo.get_best_acquisition(arr_initials, fun_objective)¶

It returns the best example with respect to values of fun_objective. Here, the best acquisition is a minimizer of fun_objective.

Parameters:	arr_initials (numpy.ndarray) – inputs. Shape: (n, d). fun_objective (function) – an objective function.
Returns:	the best example of arr_initials. Shape: (1, d).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.bo.get_grids(arr_ranges, int_grids)¶

It returns grids of given arr_ranges, where each of dimension has int_grids partitions.

Parameters:	arr_ranges (numpy.ndarray) – ranges. Shape: (d, 2). int_grids (int.) – the number of partitions per dimension.
Returns:	grids of given arr_ranges. Shape: (int_grids\(^{\text{d}}\), d).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.constants¶

This file is for declaring various default constants. If you would like to see the details, check out the repository.

bayeso.covariance¶

bayeso.covariance.choose_fun_cov(str_cov, is_grad=False)¶

It is for choosing a covariance function or a function for computing gradients of covariance function.

Parameters:	str_cov (str.) – the name of covariance function. is_grad (bool., optional) – flag for returning a function for the gradients
Returns:	covariance function, or function for computing gradients of covariance function.
Return type:	function
Raises:	AssertionError

bayeso.covariance.cov_main(str_cov, X, Xs, hyps, same_X_Xs, jitter=1e-05)¶

It computes kernel matrix over X and Xs, where hyps is given.

Parameters:	str_cov (str.) – the name of covariance function. X (numpy.ndarray) – one inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). hyps (dict.) – dictionary of hyperparameters for covariance function. same_X_Xs (bool.) – flag for checking X and Xs are same. jitter (float, optional) – jitter for diagonal entries.
Returns:	kernel matrix over X and Xs. Shape: (n, m).
Return type:	numpy.ndarray
Raises:	AssertionError, ValueError

bayeso.covariance.cov_matern32(X, Xs, lengthscales, signal)¶

It computes Matern 3/2 kernel over X and Xs, where lengthscales and signal are given.

Parameters:	X (numpy.ndarray) – inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). lengthscales (numpy.ndarray, or float) – length scales. Shape: (d, ) or (). signal (float) – coefficient for signal.
Returns:	kernel values over X and Xs. Shape: (n, m).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.cov_matern52(X, Xs, lengthscales, signal)¶

It computes Matern 5/2 kernel over X and Xs, where lengthscales and signal are given.

Parameters:	X (numpy.ndarray) – inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). lengthscales (numpy.ndarray, or float) – length scales. Shape: (d, ) or (). signal (float) – coefficient for signal.
Returns:	kernel values over X and Xs. Shape: (n, m).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.cov_se(X, Xs, lengthscales, signal)¶

It computes squared exponential kernel over X and Xs, where lengthscales and signal are given.

Parameters:	X (numpy.ndarray) – inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). lengthscales (numpy.ndarray, or float) – length scales. Shape: (d, ) or (). signal (float) – coefficient for signal.
Returns:	kernel values over X and Xs. Shape: (n, m).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.cov_set(str_cov, X, Xs, lengthscales, signal)¶

It computes set kernel matrix over X and Xs, where lengthscales and signal are given.

Parameters:	str_cov (str.) – the name of covariance function. X (numpy.ndarray) – one inputs. Shape: (n, m, d). Xs (numpy.ndarray) – another inputs. Shape: (l, m, d). lengthscales (numpy.ndarray, or float) – length scales. Shape: (d, ) or (). signal (float) – coefficient for signal.
Returns:	set kernel matrix over X and Xs. Shape: (n, l).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.grad_cov_main(str_cov, X, Xs, hyps, is_fixed_noise, same_X_Xs=True, jitter=1e-05)¶

It computes gradients of kernel matrix over hyperparameters, where hyps is given.

Parameters:	str_cov (str.) – the name of covariance function. X (numpy.ndarray) – one inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). hyps (dict.) – dictionary of hyperparameters for covariance function. is_fixed_noise (bool.) – flag for fixing a noise. same_X_Xs (bool., optional) – flag for checking X and Xs are same. jitter (float, optional) – jitter for diagonal entries.
Returns:	gradient matrix over hyperparameters. Shape: (n, m, l) where l is the number of hyperparameters.
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.grad_cov_matern32(cov_, X, Xs, hyps, num_hyps, is_fixed_noise)¶

It computes gradients of Matern 3/2 kernel over X and Xs, where hyps is given.

Parameters:	cov (numpy.ndarray) – covariance matrix. Shape: (n, m). X (numpy.ndarray) – one inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). hyps (dict.) – dictionary of hyperparameters for covariance function. num_hyps (int.) – the number of hyperparameters == l. is_fixed_noise (bool.) – flag for fixing a noise.
Returns:	gradient matrix over hyperparameters. Shape: (n, m, l).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.grad_cov_matern52(cov_, X, Xs, hyps, num_hyps, is_fixed_noise)¶

It computes gradients of Matern 5/2 kernel over X and Xs, where hyps is given.

Parameters:	cov (numpy.ndarray) – covariance matrix. Shape: (n, m). X (numpy.ndarray) – one inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). hyps (dict.) – dictionary of hyperparameters for covariance function. num_hyps (int.) – the number of hyperparameters == l. is_fixed_noise (bool.) – flag for fixing a noise.
Returns:	gradient matrix over hyperparameters. Shape: (n, m, l).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.covariance.grad_cov_se(cov_, X, Xs, hyps, num_hyps, is_fixed_noise)¶

It computes gradients of squared exponential kernel over X and Xs, where hyps is given.

Parameters:	cov (numpy.ndarray) – covariance matrix. Shape: (n, m). X (numpy.ndarray) – one inputs. Shape: (n, d). Xs (numpy.ndarray) – another inputs. Shape: (m, d). hyps (dict.) – dictionary of hyperparameters for covariance function. num_hyps (int.) – the number of hyperparameters == l. is_fixed_noise (bool.) – flag for fixing a noise.
Returns:	gradient matrix over hyperparameters. Shape: (n, m, l).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.gp¶

bayeso.gp.gp¶

bayeso.gp.gp.get_optimized_kernel(X_train, Y_train, prior_mu, str_cov, str_framework='scipy', str_optimizer_method='Nelder-Mead', str_modelselection_method='ml', is_fixed_noise=True, debug=False)¶

This function computes the kernel matrix optimized by optimization method specified, its inverse matrix, and the optimized hyperparameters.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). prior_mu (function or NoneType) – prior mean function or None. str_cov (str.) – the name of covariance function. str_framework (str.) – the name of framework for optimizing kernel hyperparameters. str_optimizer_method (str., optional) – the name of optimization method. str_modelselection_method (str., optional) – the name of model selection method. is_fixed_noise (bool., optional) – flag for fixing a noise. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of kernel matrix over X_train, kernel matrix inverse, and dictionary of hyperparameters.
Return type:	tuple of (numpy.ndarray, numpy.ndarray, dict.)
Raises:	AssertionError, ValueError

bayeso.gp.gp.predict_optimized(X_train, Y_train, X_test, str_cov='matern52', prior_mu=None, is_fixed_noise=True, debug=False)¶

This function returns posterior mean and posterior standard deviation functions over X_test, computed by the Gaussian process regression optimized with X_train and Y_train.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). X_test (numpy.ndarray) – inputs. Shape: (l, d) or (l, m, d). str_cov (str., optional) – the name of covariance function. prior_mu (NoneType, or function, optional) – None, or prior mean function. is_fixed_noise (bool., optional) – flag for fixing a noise. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of posterior mean function over X_test, posterior standard deviation function over X_test, and posterior covariance matrix over X_test. Shape: ((l, 1), (l, 1), (l, l)).
Return type:	tuple of (numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.gp.gp.predict_test(X_train, Y_train, X_test, hyps, str_cov='matern52', prior_mu=None, debug=False)¶

This function returns posterior mean and posterior standard deviation functions over X_test, computed by Gaussian process regression with X_train, Y_train, and hyps.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). X_test (numpy.ndarray) – inputs. Shape: (l, d) or (l, m, d). hyps (dict.) – dictionary of hyperparameters for Gaussian process. str_cov (str., optional) – the name of covariance function. prior_mu (NoneType, or function, optional) – None, or prior mean function. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of posterior mean function over X_test, posterior standard deviation function over X_test, and posterior covariance matrix over X_test. Shape: ((l, 1), (l, 1), (l, l)).
Return type:	tuple of (numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.gp.gp.predict_test_(X_train, Y_train, X_test, cov_X_X, inv_cov_X_X, hyps, str_cov='matern52', prior_mu=None, debug=False)¶

This function returns posterior mean and posterior standard deviation functions over X_test, computed by Gaussian process regression with X_train, Y_train, cov_X_X, inv_cov_X_X, and hyps.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). X_test (numpy.ndarray) – inputs. Shape: (l, d) or (l, m, d). cov_X_X (numpy.ndarray) – kernel matrix over X_train. Shape: (n, n). inv_cov_X_X (numpy.ndarray) – kernel matrix inverse over X_train. Shape: (n, n). hyps (dict.) – dictionary of hyperparameters for Gaussian process. str_cov (str., optional) – the name of covariance function. prior_mu (NoneType, or function, optional) – None, or prior mean function. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of posterior mean function over X_test, posterior standard deviation function over X_test, and posterior covariance matrix over X_test. Shape: ((l, 1), (l, 1), (l, l)).
Return type:	tuple of (numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.gp.gp.sample_functions(mu, Sigma, num_samples=1)¶

It samples num_samples functions from multivariate Gaussian distribution (mu, Sigma).

Parameters:	mu (numpy.ndarray) – mean vector. Shape: (n, ). Sigma (numpy.ndarray) – covariance matrix. Shape: (n, n). num_samples (int., optional) – the number of sampled functions
Returns:	sampled functions. Shape: (num_samples, n).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.gp.gp_common¶

bayeso.gp.gp_common.get_kernel_cholesky(X_train, hyps, str_cov, is_fixed_noise=True, is_gradient=False, debug=False)¶

This function computes a kernel inverse with Cholesky decomposition.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). hyps (dict.) – dictionary of hyperparameters for Gaussian process. str_cov (str.) – the name of covariance function. is_fixed_noise (bool., optional) – flag for fixing a noise. is_gradient (bool., optional) – flag for computing and returning gradients of negative log marginal likelihood. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of kernel matrix over X_train, lower matrix computed by Cholesky decomposition, and gradients of kernel matrix. If is_gradient is False, gradients of kernel matrix would be None.
Return type:	tuple of (numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.gp.gp_common.get_kernel_inverse(X_train, hyps, str_cov, is_fixed_noise=True, is_gradient=False, debug=False)¶

This function computes a kernel inverse without any matrix decomposition techniques.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). hyps (dict.) – dictionary of hyperparameters for Gaussian process. str_cov (str.) – the name of covariance function. is_fixed_noise (bool., optional) – flag for fixing a noise. is_gradient (bool., optional) – flag for computing and returning gradients of negative log marginal likelihood. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of kernel matrix over X_train, kernel matrix inverse, and gradients of kernel matrix. If is_gradient is False, gradients of kernel matrix would be None.
Return type:	tuple of (numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.gp.gp_scipy¶

bayeso.gp.gp_scipy.get_optimized_kernel(X_train, Y_train, prior_mu, str_cov, str_optimizer_method='Nelder-Mead', str_modelselection_method='ml', is_fixed_noise=True, debug=False)¶

This function computes the kernel matrix optimized by optimization method specified, its inverse matrix, and the optimized hyperparameters.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). prior_mu (function or NoneType) – prior mean function or None. str_cov (str.) – the name of covariance function. str_optimizer_method (str., optional) – the name of optimization method. str_modelselection_method (str., optional) – the name of model selection method. is_fixed_noise (bool., optional) – flag for fixing a noise. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of kernel matrix over X_train, kernel matrix inverse, and dictionary of hyperparameters.
Return type:	tuple of (numpy.ndarray, numpy.ndarray, dict.)
Raises:	AssertionError, ValueError

bayeso.gp.gp_scipy.neg_log_ml(X_train, Y_train, hyps, str_cov, prior_mu_train, is_fixed_noise=True, is_cholesky=True, is_gradient=True, debug=False)¶

This function computes a negative log marginal likelihood.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). hyps (numpy.ndarray) – hyperparameters for Gaussian process. Shape: (h, ). str_cov (str.) – the name of covariance function. prior_mu_train (numpy.ndarray) – the prior values computed by get_prior_mu(). Shape: (n, 1). is_fixed_noise (bool., optional) – flag for fixing a noise. is_cholesky (bool., optional) – flag for using a cholesky decomposition. is_gradient (bool., optional) – flag for computing and returning gradients of negative log marginal likelihood. debug (bool., optional) – flag for printing log messages.
Returns:	negative log marginal likelihood, or (negative log marginal likelihood, gradients of the likelihood).
Return type:	float, or tuple of (float, float)
Raises:	AssertionError

bayeso.gp.gp_scipy.neg_log_pseudo_l_loocv(X_train, Y_train, hyps, str_cov, prior_mu_train, is_fixed_noise=True, debug=False)¶

It computes a negative log pseudo-likelihood using leave-one-out cross-validation.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). hyps (numpy.ndarray) – hyperparameters for Gaussian process. Shape: (h, ). str_cov (str.) – the name of covariance function. prior_mu_train (numpy.ndarray) – the prior values computed by get_prior_mu(). Shape: (n, 1). is_fixed_noise (bool., optional) – flag for fixing a noise. debug (bool., optional) – flag for printing log messages.
Returns:	negative log pseudo-likelihood.
Return type:	float
Raises:	AssertionError

bayeso.gp.gp_tensorflow¶

bayeso.gp.gp_tensorflow.get_optimized_kernel(X_train, Y_train, prior_mu, str_cov, is_fixed_noise=True, num_iters=1000, debug=False)¶

This function computes the kernel matrix optimized by optimization method specified, its inverse matrix, and the optimized hyperparameters, using TensorFlow and TensorFlow probability.

Parameters:	X_train (numpy.ndarray) – inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – outputs. Shape: (n, 1). prior_mu (function or NoneType) – prior mean function or None. str_cov (str.) – the name of covariance function. is_fixed_noise (bool., optional) – flag for fixing a noise. num_iters (int., optional) – the number of iterations for optimizing negative log likelihood. debug (bool., optional) – flag for printing log messages.
Returns:	a tuple of kernel matrix over X_train, kernel matrix inverse, and dictionary of hyperparameters.
Return type:	tuple of (numpy.ndarray, numpy.ndarray, dict.)
Raises:	AssertionError, ValueError

bayeso.utils¶

bayeso.utils.utils_bo¶

bayeso.utils.utils_bo.get_next_best_acquisition(arr_points, arr_acquisitions, cur_points)¶

It returns the next best acquired example.

Parameters:	arr_points (numpy.ndarray) – inputs for acquisition function. Shape: (n, d). arr_acquisitions (numpy.ndarray) – acquisition function values over arr_points. Shape: (n, ). cur_points (numpy.ndarray) – examples evaluated so far. Shape: (m, d).
Returns:	next best acquired point. Shape: (d, ).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.utils.utils_bo.optimize_many(model_bo, fun_target, X_train, int_iter, str_initial_method_ao='uniform', int_samples_ao=100, str_mlm_method='regular')¶

It optimizes fun_target for int_iter iterations with given model_bo and initial inputs X_train. It returns the optimization results and execution times.

Parameters:	model_bo (bayeso.bo.BO) – Bayesian optimization model. fun_target (function) – a target function. X_train (numpy.ndarray) – initial inputs. Shape: (n, d) or (n, m, d). int_iter (int.) – the number of iterations for Bayesian optimization. str_initial_method_ao (str., optional) – the name of initialization method for acquisition function optimization. int_samples_ao (int., optional) – the number of samples for acquisition function optimization. If L-BFGS-B is used as an acquisition function optimization method, it is employed. str_mlm_method (str., optional) – the name of marginal likelihood maximization method for Gaussian process regression.
Returns:	tuple of acquired examples, their function values, overall execution times per iteration, execution time consumed in Gaussian process regression, and execution time consumed in acquisition function optimization. Shape: ((n + int_iter, d), (n + int_iter, 1), (n + int_iter, ), (int_iter, ), (int_iter, )), or ((n + int_iter, m, d), (n + int_iter, m, 1), (n + int_iter, ), (int_iter, ), (int_iter, )).
Return type:	(numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.utils.utils_bo.optimize_many_(model_bo, fun_target, X_train, Y_train, int_iter, str_initial_method_ao='uniform', int_samples_ao=100, str_mlm_method='regular')¶

It optimizes fun_target for int_iter iterations with given model_bo. It returns the optimization results and execution times.

Parameters:	model_bo (bayeso.bo.BO) – Bayesian optimization model. fun_target (function) – a target function. X_train (numpy.ndarray) – initial inputs. Shape: (n, d) or (n, m, d). Y_train (numpy.ndarray) – initial outputs. Shape: (n, 1). int_iter (int.) – the number of iterations for Bayesian optimization. str_initial_method_ao (str., optional) – the name of initialization method for acquisition function optimization. int_samples_ao (int., optional) – the number of samples for acquisition function optimization. If L-BFGS-B is used as an acquisition function optimization method, it is employed. str_mlm_method (str., optional) – the name of marginal likelihood maximization method for Gaussian process regression.
Returns:	tuple of acquired examples, their function values, overall execution times per iteration, execution time consumed in Gaussian process regression, and execution time consumed in acquisition function optimization. Shape: ((n + int_iter, d), (n + int_iter, 1), (int_iter, ), (int_iter, ), (int_iter, )), or ((n + int_iter, m, d), (n + int_iter, m, 1), (int_iter, ), (int_iter, ), (int_iter, )).
Return type:	(numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.utils.utils_bo.optimize_many_with_random_init(model_bo, fun_target, int_init, int_iter, str_initial_method_bo='uniform', str_initial_method_ao='uniform', int_samples_ao=100, str_mlm_method='regular', int_seed=None)¶

It optimizes fun_target for int_iter iterations with given model_bo and int_init initial examples. Initial examples are sampled by get_initial method in model_bo. It returns the optimization results and execution times.

Parameters:	model_bo (bayeso.bo.BO) – Bayesian optimization model. fun_target (function) – a target function. int_init (int.) – the number of initial examples for Bayesian optimization. int_iter (int.) – the number of iterations for Bayesian optimization. str_initial_method_bo (str., optional) – the name of initialization method for sampling initial examples in Bayesian optimization. str_initial_method_ao (str., optional) – the name of initialization method for acquisition function optimization. int_samples_ao (int., optional) – the number of samples for acquisition function optimization. If L-BFGS-B is used as an acquisition function optimization method, it is employed. str_mlm_method (str., optional) – the name of marginal likelihood maximization method for Gaussian process regression. int_seed (NoneType or int., optional) – None, or random seed.
Returns:	tuple of acquired examples, their function values, overall execution times per iteration, execution time consumed in Gaussian process regression, and execution time consumed in acquisition function optimization. Shape: ((int_init + int_iter, d), (int_init + int_iter, 1), (int_init + int_iter, ), (int_iter, ), (int_iter, )), or ((int_init + int_iter, m, d), (int_init + int_iter, m, 1), (int_init + int_iter, ), (int_iter, ), (int_iter, )), where d is a dimensionality of the problem we are solving and m is a cardinality of sets.
Return type:	(numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.utils.utils_common¶

bayeso.utils.utils_common.get_minimum(data_all, int_init)¶

It returns accumulated minima at each iteration, their arithmetic means over rounds, and their standard deviations over rounds, which is widely used in Bayesian optimization community.

Parameters:	data_all (numpy.ndarray) – historical function values. Shape: (r, t) where r is the number of Bayesian optimization rounds and t is the number of iterations including initial points for each round. For example, if we run 50 iterations with 5 initial examples and repeat this procedure 3 times, r would be 3 and t would be 55 (= 50 + 5). int_init (int.) – the number of initial points.
Returns:	tuple of accumulated minima, their arithmetic means over rounds, and their standard deviations over rounds. Shape: ((r, t - int_init + 1), (t - int_init + 1, ), (t - int_init + 1, )).
Return type:	(numpy.ndarray, numpy.ndarray, numpy.ndarray)
Raises:	AssertionError

bayeso.utils.utils_common.get_time(arr_time, int_init, is_initial)¶

It returns the means of accumulated execution times over rounds.

Parameters:	arr_time (numpy.ndarray) – execution times for all Bayesian optimization rounds. Shape: (r, t) where r is the number of Bayesian optimization rounds and t is the number of iterations (including initial points if is_initial is True, or excluding them if is_initial is False) for each round. int_init (int.) – the number of initial points. If is_initial is False, it is ignored even if it is provided. is_initial (bool.) – flag for describing whether execution times to observe initial examples have been included or not.
Returns:	arithmetic means of accumulated execution times over rounds. Shape: (t - int_init, ) if is_initial is True. (t, ), otherwise.
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.utils.utils_covariance¶

bayeso.utils.utils_covariance._get_list_first()¶

It provides list of strings. The strings in that list require two hyperparameters, signal and lengthscales. We simply call it as list_first.

Returns:	list of strings, which satisfy some requirements we mentioned above.
Return type:	list

bayeso.utils.utils_covariance.convert_hyps(str_cov, hyps, is_fixed_noise=False)¶

It converts hyperparameters dictionary, hyps to numpy array.

Parameters:	str_cov (str.) – the name of covariance function. hyps (dict.) – dictionary of hyperparameters for covariance function. is_fixed_noise (bool., optional) – flag for fixing a noise.
Returns:	converted array of the hyperparameters given by hyps.
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.utils.utils_covariance.get_hyps(str_cov, int_dim, is_ard=True)¶

It returns a dictionary of default hyperparameters for covariance function, where str_cov and int_dim are given. If is_ard is True, the length scales would be int_dim-dimensional vector.

Parameters:	str_cov (str.) – the name of covariance function. int_dim (int.) – dimensionality of the problem we are solving. is_ard (bool., optional) – flag for automatic relevance determination.
Returns:	dictionary of default hyperparameters for covariance function.
Return type:	dict.
Raises:	AssertionError

bayeso.utils.utils_covariance.get_range_hyps(str_cov, int_dim, is_ard=True, is_fixed_noise=False)¶

It returns default optimization ranges of hyperparameters for Gaussian process regression.

Parameters:	str_cov (str.) – the name of covariance function. int_dim (int.) – dimensionality of the problem we are solving. is_ard (bool., optional) – flag for automatic relevance determination. is_fixed_noise (bool., optional) – flag for fixing a noise.
Returns:	list of default optimization ranges for hyperparameters.
Return type:	list
Raises:	AssertionError

bayeso.utils.utils_covariance.restore_hyps(str_cov, hyps, is_fixed_noise=False, fixed_noise=0.01)¶

It restores hyperparameters array, hyps to dictionary.

Parameters:	str_cov (str.) – the name of covariance function. hyps (numpy.ndarray) – array of hyperparameters for covariance function. is_fixed_noise (bool., optional) – flag for fixing a noise. fixed_noise (float, optional) – fixed noise value.
Returns:	restored dictionary of the hyperparameters given by hyps.
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.utils.utils_covariance.validate_hyps_arr(arr_hyps, str_cov, int_dim)¶

It validates hyperparameters array, arr_hyps.

Parameters:	arr_hyps (numpy.ndarray) – array of hyperparameters for covariance function. str_cov (str.) – the name of covariance function. int_dim (int.) – dimensionality of the problem we are solving.
Returns:	a tuple of valid hyperparameters and validity flag.
Return type:	(numpy.ndarray, bool.)
Raises:	AssertionError

bayeso.utils.utils_covariance.validate_hyps_dict(dict_hyps, str_cov, int_dim)¶

It validates hyperparameters dictionary, dict_hyps.

Parameters:	dict_hyps (dict.) – dictionary of hyperparameters for covariance function. str_cov (str.) – the name of covariance function. int_dim (int.) – dimensionality of the problem we are solving.
Returns:	a tuple of valid hyperparameters and validity flag.
Return type:	(dict., bool.)
Raises:	AssertionError

bayeso.utils.utils_gp¶

bayeso.utils.utils_gp.check_str_cov(str_fun, str_cov, shape_X1, shape_X2=None)¶

It is for validating the shape of X1 (and optionally the shape of X2).

Parameters:	str_fun (str.) – the name of function. str_cov (str.) – the name of covariance function. shape_X1 (tuple) – the shape of X1. shape_X2 (NoneType or tuple, optional) – None, or the shape of X2.
Returns:	None, if it is valid. Raise an error, otherwise.
Return type:	NoneType
Raises:	AssertionError, ValueError

bayeso.utils.utils_gp.get_prior_mu(prior_mu, X)¶

It computes the prior mean function values over inputs X.

Parameters:	prior_mu (function or NoneType) – prior mean function or None. X (numpy.ndarray) – inputs for prior mean function. Shape: (n, d) or (n, m, d).
Returns:	zero array, or array of prior mean function values. Shape: (n, 1).
Return type:	numpy.ndarray
Raises:	AssertionError

bayeso.utils.utils_plotting¶

bayeso.utils.utils_plotting._save_figure(path_save, str_postfix, str_prefix='')¶

It saves a figure.

Parameters:	path_save (str.) – path for saving a figure. str_postfix (str.) – the name of postfix. str_prefix (str., optional) – the name of prefix.
Returns:	None.
Return type:	NoneType

bayeso.utils.utils_plotting._set_ax_config(ax, str_x_axis, str_y_axis, size_labels=32, size_ticks=22, xlim_min=None, xlim_max=None, is_box=True, is_zero_axis=False, is_grid=True)¶

It sets an axis configuration.

Parameters:	ax (matplotlib.axes._subplots.AxesSubplot) – inputs for acquisition function. Shape: (n, d). str_x_axis (str.) – the name of x axis. str_y_axis (str.) – the name of y axis. size_labels (int., optional) – label size. size_ticks (int., optional) – tick size. xlim_min (NoneType or float, optional) – None, or minimum for x limit. xlim_max (NoneType or float, optional) – None, or maximum for x limit. is_box (bool., optional) – flag for drawing a box. is_zero_axis (bool., optional) – flag for drawing a zero axis. is_grid (bool., optional) – flag for drawing grids.
Returns:	None.
Return type:	NoneType

bayeso.utils.utils_plotting._set_font_config(is_tex)¶

It sets a font configuration.

Parameters:	is_tex (bool.) – flag for using latex.
Returns:	None.
Return type:	NoneType

bayeso.utils.utils_plotting._show_figure(is_pause, time_pause)¶

It shows a figure.

Parameters:	is_pause (bool.) – flag for pausing before closing a figure. time_pause (float) – pausing time.
Returns:	None.
Return type:	NoneType

bayeso.utils.utils_plotting.plot_bo_step(X_train, Y_train, X_test, Y_test, mean_test, std_test, path_save=None, str_postfix=None, str_x_axis='x', str_y_axis='y', int_init=None, is_tex=False, is_zero_axis=False, is_pause=True, time_pause=2.0, range_shade=1.96)¶

It is for plotting Bayesian optimization results step by step.

Parameters:	X_train (numpy.ndarray) – training inputs. Shape: (n, 1). Y_train (numpy.ndarray) – training outputs. Shape: (n, 1). X_test (numpy.ndarray) – test inputs. Shape: (m, 1). Y_test (NoneType or numpy.ndarray, optional) – None, or true test outputs. Shape: (m, 1). mean_test (numpy.ndarray) – posterior predictive mean function values over X_test. Shape: (m, 1). std_test (numpy.ndarray) – posterior predictive standard deviation function values over X_test. Shape: (m, 1). path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. int_init (NoneType or int., optional) – None, or the number of initial examples. is_tex (bool., optional) – flag for using latex. is_zero_axis (bool., optional) – flag for drawing a zero axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. range_shade (float, optional) – shade range for standard deviation.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso.utils.utils_plotting.plot_bo_step_acq(X_train, Y_train, X_test, Y_test, mean_test, std_test, acq_test, path_save=None, str_postfix=None, str_x_axis='x', str_y_axis='y', str_acq_axis='acq.', int_init=None, is_tex=False, is_zero_axis=False, is_pause=True, time_pause=2.0, range_shade=1.96)¶

It is for plotting Bayesian optimization results step by step.

Parameters:	X_train (numpy.ndarray) – training inputs. Shape: (n, 1). Y_train (numpy.ndarray) – training outputs. Shape: (n, 1). X_test (numpy.ndarray) – test inputs. Shape: (m, 1). Y_test (NoneType or numpy.ndarray, optional) – None, or true test outputs. Shape: (m, 1). mean_test (numpy.ndarray) – posterior predictive mean function values over X_test. Shape: (m, 1). std_test (numpy.ndarray) – posterior predictive standard deviation function values over X_test. Shape: (m, 1). acq_test (numpy.ndarray) – acquisition funcion values over X_test. Shape: (m, 1). path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. str_acq_axis (str., optional) – the name of acquisition function axis. int_init (NoneType or int., optional) – None, or the number of initial examples. is_tex (bool., optional) – flag for using latex. is_zero_axis (bool., optional) – flag for drawing a zero axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. range_shade (float, optional) – shade range for standard deviation.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso.utils.utils_plotting.plot_gp(X_train, Y_train, X_test, mu, sigma, Y_test_truth=None, path_save=None, str_postfix=None, str_x_axis='x', str_y_axis='y', is_tex=False, is_zero_axis=False, is_pause=True, time_pause=2.0, range_shade=1.96, colors=['red', 'green', 'blue', 'orange', 'olive', 'purple', 'darkred', 'limegreen', 'deepskyblue', 'lightsalmon', 'aquamarine', 'navy', 'rosybrown', 'darkkhaki', 'darkslategray'])¶

It is for plotting Gaussian process regression.

Parameters:	X_train (numpy.ndarray) – training inputs. Shape: (n, 1). Y_train (numpy.ndarray) – training outputs. Shape: (n, 1). X_test (numpy.ndarray) – test inputs. Shape: (m, 1). mu (numpy.ndarray) – posterior predictive mean function values over X_test. Shape: (m, 1). sigma (numpy.ndarray) – posterior predictive standard deviation function values over X_test. Shape: (m, 1). Y_test_truth (NoneType or numpy.ndarray, optional) – None, or true test outputs. Shape: (m, 1). path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. is_tex (bool., optional) – flag for using latex. is_zero_axis (bool., optional) – flag for drawing a zero axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. range_shade (float, optional) – shade range for standard deviation. colors (list, optional) – list of colors.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso.utils.utils_plotting.plot_gp_sampled(X, Ys, path_save=None, str_postfix=None, str_x_axis='x', str_y_axis='y', is_tex=False, is_zero_axis=False, is_pause=True, time_pause=2.0, colors=['red', 'green', 'blue', 'orange', 'olive', 'purple', 'darkred', 'limegreen', 'deepskyblue', 'lightsalmon', 'aquamarine', 'navy', 'rosybrown', 'darkkhaki', 'darkslategray'])¶

It is for plotting sampled functions from multivariate distributions.

Parameters:	X (numpy.ndarray) – training inputs. Shape: (n, 1). Ys (numpy.ndarray) – training outputs. Shape: (m, n). path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. is_tex (bool., optional) – flag for using latex. is_zero_axis (bool., optional) – flag for drawing a zero axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. colors (list, optional) – list of colors.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso.utils.utils_plotting.plot_minimum(arr_minima, list_str_label, int_init, is_std, is_marker=True, is_legend=False, is_tex=False, path_save=None, str_postfix=None, str_x_axis='Iteration', str_y_axis='Minimum function value', is_pause=True, time_pause=2.0, range_shade=1.96, markers=['.', 'x', '*', '+', '^', 'v', '<', '>', 'd', ',', '8', 'h', '1', '2', '3'], colors=['red', 'green', 'blue', 'orange', 'olive', 'purple', 'darkred', 'limegreen', 'deepskyblue', 'lightsalmon', 'aquamarine', 'navy', 'rosybrown', 'darkkhaki', 'darkslategray'])¶

It is for plotting optimization results of Bayesian optimization, in terms of iterations.

Parameters:	arr_minima (numpy.ndarray) – function values over acquired examples. Shape: (b, r, n) where b is the number of experiments, r is the number of rounds, and n is the number of iterations per round. list_str_label (list) – list of label strings. Shape: (b, ). int_init (int.) – the number of initial examples < n. is_std (bool.) – flag for drawing standard deviations. is_marker (bool., optional) – flag for drawing markers. is_legend (bool., optional) – flag for drawing a legend. is_tex (bool., optional) – flag for using latex. path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. range_shade (float, optional) – shade range for standard deviation. markers (list, optional) – list of markers. colors (list, optional) – list of colors.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso.utils.utils_plotting.plot_minimum_time(arr_times, arr_minima, list_str_label, int_init, is_std, is_marker=True, is_legend=False, is_tex=False, path_save=None, str_postfix=None, str_x_axis='Time (sec.)', str_y_axis='Minimum function value', is_pause=True, time_pause=2.0, range_shade=1.96, markers=['.', 'x', '*', '+', '^', 'v', '<', '>', 'd', ',', '8', 'h', '1', '2', '3'], colors=['red', 'green', 'blue', 'orange', 'olive', 'purple', 'darkred', 'limegreen', 'deepskyblue', 'lightsalmon', 'aquamarine', 'navy', 'rosybrown', 'darkkhaki', 'darkslategray'])¶

It is for plotting optimization results of Bayesian optimization, in terms of execution time.

Parameters:	arr_times (numpy.ndarray) – execution times. Shape: (b, r, n), or (b, r, int_init + n) where b is the number of experiments, r is the number of rounds, and n is the number of iterations per round. arr_minima (numpy.ndarray) – function values over acquired examples. Shape: (b, r, int_init + n) where b is the number of experiments, r is the number of rounds, and n is the number of iterations per round. list_str_label (list) – list of label strings. Shape: (b, ). int_init (int.) – the number of initial examples. is_std (bool.) – flag for drawing standard deviations. is_marker (bool., optional) – flag for drawing markers. is_legend (bool., optional) – flag for drawing a legend. is_tex (bool., optional) – flag for using latex. path_save (NoneType or str., optional) – None, or path for saving a figure. str_postfix (NoneType or str., optional) – None, or the name of postfix. str_x_axis (str., optional) – the name of x axis. str_y_axis (str., optional) – the name of y axis. is_pause (bool., optional) – flag for pausing before closing a figure. time_pause (float, optional) – pausing time. range_shade (float, optional) – shade range for standard deviation. markers (list, optional) – list of markers. colors (list, optional) – list of colors.
Returns:	None.
Return type:	NoneType
Raises:	AssertionError

bayeso: A Bayesian optimization framework in Python¶

About bayeso¶

Supported Python Version¶

Related Package for Benchmark Functions¶

Contributor¶

Citation¶

Contact¶

License¶

About Bayesian Optimization¶

Installing bayeso¶

Installing from PyPI¶

Compiling from Source¶

Compiling from Source (Editable)¶

Uninstalling¶

Required Packages¶

Building Gaussian Process Regression¶

Optimizing Sampled Function via Thompson Sampling¶

Optimizing Branin Function¶

Constructing xgboost Classifier with Hyperparameter Optimization¶

bayeso¶

bayeso.acquisition¶

bayeso.bo¶

bayeso.constants¶

bayeso.covariance¶

bayeso.gp¶

bayeso.gp.gp¶

bayeso.gp.gp_common¶

bayeso.gp.gp_scipy¶

bayeso.gp.gp_tensorflow¶

bayeso.utils¶

bayeso.utils.utils_bo¶

bayeso.utils.utils_common¶

bayeso.utils.utils_covariance¶

bayeso.utils.utils_gp¶

bayeso.utils.utils_plotting¶