About Bayesian Optimization

Bayesian optimization is a global optimization strategy for black-box and expensive-to-evaluate functions. Generic Bayesian optimization follows these steps:

1. Build a surrogate function with historical inputs and their observations.
2. Compute and maximize an acquisition function, defined by the outputs of surrogate function.
3. Observe the maximizer of acquisition function from a true objective function.
4. Accumulate the maximizer and its observation.

This project helps us to execute this Bayesian optimization procedure. In particular, Gaussian process regression is used as a surrogate function, and various acquisition functions such as probability improvement, expected improvement, and Gaussian process upper confidence bound are included in this project.