Softmax

What is Softmax?

The Softmax function converts predicted values to be between 0 and 1, ensuring that the sum of all predicted values is 1. For example, consider a CNN with predicted values.

As you see in the image above, the sum of the two predicted values is 1. Even though the two output nodes aren’t connected, how is the sum of the two predicted values 1?” It’s because we apply the Softmax function at the final layer of the CNN. The formula for the Softmax function is:

$f_{j}(z)\ =\ \frac{e^{zj}}{\sum_{k}e^{zk}} $

Additionally, cross-entropy is used when calculating the error rate.