Probabilistic_Layers_Non_Linear_Regression_v1.ipynb

probabilistic_layers_non_linear_regression_v1.ipynb

I would like to thank you for sharing this helpful code , I have question how we can do this in case of classification ?

I did not give it a try yet but it should work with the use of the tfp.distributions.Categorical distribution. Let me know how it went.

Thank you for your reply , actually it my first probabilistic model and I want to build a model for binary classification that provides uncertainty .
How can I code this in tensorflow ?
how can we plot uncertainty results if we have more than one dimension ( independent variables )?

For classification, I would probably replace the normal distribution in the last layer with the categorical distribution. During inference, you can then calculate multiple predictions and multiply the probabilities of each prediction element-wise and finally taking the argmax and the respective probability as certainty. For comparability with other models it could make sense to normalize the certainty by 1/k where k is the number of categories: certainty = (prob - 1/k) / (1 - 1/k). Then the certainty is 0 in case of maximal uncertainty, meaning that all categories have the same probability and 1 at maximal certainty.

Regarding your second question I am not sure what you want to do. It depends on what you want to emphasize. If you are still referring to classification with multiple labels, you could just plot the top matches with their certainty/uncertainty. In case of multi head models I would usually analyze each head separately.

Thank you so much for your great explanations, it helpful for me. Congratulations for your posting.

I gave it a bit more thought and I think you should adjust my earlier suggestion in the following way:

During inference, one can then calculate multiple predictions and multiply the probabilities of the predictions element-wise. Finally one can take the argmax as final prediction.
As the decision boundary lies between the most and the second most probable class the distance between these two should be a measurement of certainty for the selected class. Finally I would normalize by the highest probability:

certainty = (p_1st - p_2nd) / (p_1st + epsilon)