AdaGrad is an adaptive gradient-based optimization algorithm that adjusts learning rates separately for individual model parameters. It increases sensitivity to parameter-specific update histories, which can help with sparse features but may reduce learning rates substantially over time.
A sophisticated gradient descent algorithm that rescales the gradients of each parameter, effectively giving each parameter an independent learning rate. For a full explanation, see Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.