Candidate sampling is a training-time optimization that evaluates all positive labels while using only a sampled subset of negative labels. It reduces computational cost in classification problems with many possible classes while preserving learning signal for the correct classes.
A training-time optimization that calculates a probability for all the positive labels, using, for example, softmax, but only for a random sample of negative labels. For instance, given an example labeled beagle and dog, candidate sampling computes the predicted probabilities and corresponding loss terms for: - beagle - dog - a random subset of the remaining negative classes (for example, cat , lollipop , fence). The idea is that the negative classes can learn from less frequent negative reinforcement as long as positive classes always get proper positive reinforcement, and this is indeed observed empirically. Candidate sampling is more computationally efficient than training algorithms that compute predictions for all negative classes, particularly when the number of negative classes is very large.