The exploding gradient problem occurs when gradients in a neural network become excessively large during training. This can cause unstable weight updates and make a model difficult or impossible to train, with gradient clipping commonly used as a mitigation.
The tendency for gradients in deep neural networks (especially recurrent neural networks) to become surprisingly steep (high). Steep gradients often cause very large updates to the weights of each node in a deep neural network. Models suffering from the exploding gradient problem become difficult or impossible to train. Gradient clipping can mitigate this problem. Compare to vanishing gradient problem.