Softmax Function

The softmax function is an activation function that converts a vector of values into a probability distribution. It is commonly used in multi-class classification tasks to represent the probabilities of each class.

Given an input vector $z=[z_1,z_2,\dots ,z_K]$, where each $z_j$ is the raw score (the $z_j$ values are also called logits) for class $j$, the softmax function outputs:

$\sigma (z{)}_j=\frac{e^{z_j}}{\sum {k=1}^K{e}^{z_k}}$ where:

• $\sigma (z{)}_j$  is the probability of class  $j$;
• $K$  is the total number of classes;
• $e^{z_j}$  exponentiates the score for each class to ensure non-negativity.

The resulting probabilities  $\sigma (z)=[\sigma (z{)}_1,\sigma (z{)}_2,\dots ,\sigma (z{)}_K]$ sum to 1, making softmax ideal for representing mutually exclusive class probabilities.

Note

Usually this function is used as the last layer to a Neural Network to ensure the probability vector sums to $1$.

---

machine-learning