Caesar AI Atlas

Learning Rate

Caesar AI Atlas Definition

Learning rate is a hyperparameter that controls the size of parameter updates during model training. If it is too high, training may become unstable; if it is too low, training may be slow or get stuck before reaching good performance.

Other Definitions

Learning Rate Source

A floating-point number that tells the gradient descent algorithm how strongly to adjust weights and biases on each iteration. For example, a learning rate of 0.3 would adjust weights and biases three times more powerfully than a learning rate of 0.1. Learning rate is a key hyperparameter. If you set the learning rate too low, training will take too long. If you set the learning rate too high, gradient descent often has trouble reaching convergence. During each iteration, the gradient descent algorithm multiplies the learning rate by the gradient. The resulting product is called the gradient step. See Linear regression: Hyperparameters in Machine Learning Crash Course for more information.

Also Referenced In

Related Terms