An optimizer is an algorithm or implementation used to update modelparameters during training. Common optimizers, such as AdaGrad and Adam, are variants of gradient-based methods designed to improve learning efficiency and stability.
A specific implementation of the gradient descentalgorithm. Popular optimizers include:
- AdaGrad, which stands for ADAptive GRADient descent.
- Adam, which stands for ADAptive with Momentum.