An iteration is one repeated step or cycle in a computational, training, or optimization process. In machine learning, it may refer to one update cycle of an algorithm, one pass through a loop, or one stage of refining a model or system.
A single update of a model's parameters---the model's weights and biases---during training. The batch size determines how many examples the model processes in a single iteration. For instance, if the batch size is 20, then the model processes 20 examples before adjusting the parameters. When training a neural network, a single iteration involves the following two passes: 1. A forward pass to evaluate loss on a single batch. 2. A backward pass (backpropagation) to adjust the model's parameters based on the loss and the learning rate. See Gradient descent in Machine Learning Crash Course for more information.