aka Multi-label vs Single-label
Multi-target = "checkboxes": each option (class) can be chosen or not, independently of the others Single-target model = "radio buttons": choose one and only one of the options (classes)
If we want to create 0/1 (present/absent) predictions from raw scores for each class, the way we choose 0 or 1 for each class depends on if the model is single target or multi target. For a single-target model, we always choose one class to be 1 and the rest to be 0, so we'll simply assign the class with the highest score a 1 and everything else a 0. In a multi-target model, we would instead choose a threshold (for all classes, or one threshold per class) and assign 1s if scores are above the threshold, and 0s if scores are below the threshold.
- An activation layer is a mathematical function that you apply to the numerical outputs of your machine learning model
- The "sigmoid" and "softmax" activation layers go hand-in-hand with Multi-target and Single-Target models:
- sigmoid: for each class, generate an independent score between 0-1 (given values from -inf to +inf)
- softmax: distribute 100% among all classes based on how high their scores are relative to eachother. The sum of the resulting scores across all classes should be 1.0.