Softmax is a function that converts a vector of raw scores into a probability distribution over classes. The resulting values are nonnegative and sum to 1, making softmax common in multi-class classification models.
A function that determines probabilities for each possible class in a multi-class classification model. The probabilities add up to exactly 1.0. For example, the following table shows how softmax distributes various probabilities: Probability | ---| .85 | .13 | .02 | Softmax is also called full softmax. Contrast with candidate sampling. The softmax equation is as follows: where: - $\\\_i$ is the output vector. Each element of the output vector specifies the probability of this element. The sum of all the elements in the output vector is 1.0. The output vector contains the same number of elements as the input vector, $z$. - $z$ is the input vector. Each element of the input vector contains a floating-point value. - $K$ is the number of elements in the input vector (and the output vector). For example, suppose the input vector is: Therefore, softmax calculates the denominator as follows: The softmax probability of each element is therefore: So, the output vector is therefore: The sum of the three elements in $\\\$ is 1.0. Phew! See Neural networks: Multi-class classification in Machine Learning Crash Course for more information.