Also known as: Bayesian Optimisation
Bayesian optimization is a strategy for optimizing objective functions that are expensive to evaluate. It builds a probabilistic surrogate model of the objective and uses uncertainty estimates to choose promising evaluations. It is commonly used for hyperparameter tuning and other low-sample optimization tasks.
A probabilistic regression model technique for optimizing computationally expensive objective functions by instead optimizing a surrogate that quantifies the uncertainty using a Bayesian learning technique. Since Bayesian optimization is itself very expensive, it is usually used to optimize expensive-to-evaluate tasks that have a small number of parameters, such as selecting hyperparameters.