Hill climbing is an iterative optimization method that starts from an initial solution and repeatedly adopts nearby changes that improve performance. It can be used for model tuning or search problems, but it may stop at a local optimum rather than the best possible global solution.
An algorithm for iteratively improving ("walking uphill") an ML model until the model stops improving ("reaches the top of a hill"). The general form of the algorithm is as follows: 1. Build a starting model. 2. Create new candidate models by making small adjustments to the way you train or fine-tune. This might entail working with a slightly different training set or different hyperparameters. 3. Evaluate the new candidate models and take one of the following actions: - If a candidate model outperforms the starting model, then that candidate model becomes the new starting model. In this case, repeat Steps 1, 2, and 3. - If no model outperforms the starting model, then you've reached the top of the hill and should stop iterating. [!NOTE] Note: Think of the top of the hill as a local maximum that isn't necessarily a global maximum. That is, hill climbing can help you find the best model within your current constraints. However, you might be able to build an even better model by starting over with a new approach. See Deep Learning Tuning Playbook for guidance on hyperparameter tuning. See the Data modules of Machine Learning Crash Course for guidance on feature engineering.