• Categorical hinge : Mainly for SVM soft margins

• Binary cross-entropy : for 2 class only

• Categorical cross-entropy: Multi class but not necessarily Mutually exclusive

• Sparse categorical cross-entropy : Multi class + Mutually exclusive only , saves memory too

• Categorical_crossentropy (cce) produces a one-hot array containing the probable match for each category,

• Sparse_categorical_crossentropy (scce) produces a category index of the most likely matching category.

---------------

Many categorical models produce scce output because you save space, but lose A LOT of information - generally prefer cce output for model reliability.

There are a number of situations to use scce, including:

•  When your classes are mutually exclusive, i.e. you don't care at all about other close-enough predictions,
• the number of categories is large to the prediction output becomes overwhelming.