A simple approach that uses sampling from either probability distributions or approximations of distributions as input and output from deterministic functions.
As examples, we'll consider observations and predictions. A prediction is computed from a set of observations. We'll work with observations and predictions of probability distributions of tuples .
A prediction function is a function from a set of observations to a prediction
. This function doesn't take distributions as input, nor does it produce a distribution as output. This function for now is just a naive function from a set of tuples to a tuple.
An uncertain observation is a probability distribution . Same definition for an uncertain prediction.
Now generalize our prediction function to a function with an uncertain prediction as output (while still taking plain tuples as input): .
To represent distributions, we can perhaps use some probability distributions from ProbOnto. In addition, we might want to use approximations. For example:
(This specific representation assumes numeric tuple components. Variations for categorical components are easy.)
We need to be able to sample from all distributions.
We now compute a prediction based on uncertain observations by sampling from those distributions, applying the prediction function, and then sampling from the output distribution (if the prediction function outputs a non-trivial distribution).
Our knowledge representation can look like
or
Of course, you could reorganize this representations in several plausible ways. For example, you could reify and annotate the observations
o isA Observation
o createdAt 2018-05-01T12:34:56.789Z
o subject a
o observation Gaussian(mean=<x,y,z>,stddev=<x',y',z'>)
- Good: The prediction function is arbitrary. In particular, it does not have to be distribution-aware. It can output a (non-trivial) distribution.
- Bad: Your prediction function could -- by its nature -- introduce or amplify sampling-related trouble.
- Bad: To get acceptable accuracy, the size of the representation of the distribution approximation could be very large. In some cases, the required accuracy might not be feasible to represent (or at least not without fancy gear).
- Good: You can use proper distributions when you have them.
- Good: You can control the cost/accuracy of the approximations of the output distribution.\
- Bad: Sampling can be tricky -- and much harder than it sounds.
- Good: The algorithm is simple, and its semantics are relatively clear. We are sampling. Other than potentially fancy sampling techniques, you don't need high-powered math.
- Bad: You'd get much better results with much, much less computation if you were working with distributions all the way through (including, of course, in the prediction function).
- Good: If you have GPUs, you might be able to very fast.
- Good: You have access to those counts (when they are really counts and not probabilities) in the approximate distributions.