Hi,
I am working on some regression problem and I have a question about implementation of algorithm where as an input we don’t have a single number, because we know it only with some standard deviation (or more with the distribution similar to gaussian).
For example as a label I don’t have: 1 but the maximum is 1 and minimums are in 0.5 and 1.5.
Is it possible (and if yes- how) to pass an input like a distribution (for example gaussian) or a number with uncertainty. As an output we can have a single number.
Anyone?
Sure, just train on random samples, e.g. x=torch.distributions.Normal(m,s).rsample()
But then I need to train every element of data few times (as much as number of samples). I mean for every label e.g. 1 I will make distribution built of 100 elements, so also I need to pass labeled data 100 times for every step of iteration, so for example if I work on pictures datasets - every single picture will be processed 100 times with random label. Am I right?
As you are already doing stochastic optimization, using one sample per element per iteration is not too different from optimizing usual static inputs - every new element (random dataset row) feeds a different value to the same input “slot” anyway. It is true that you may need more epochs due to sampling.
There is an approach to generate multiple samples, also called “particles”, and process them together, like an expanded batch, but it is mainly used with probabilistic/variational modeling, where densities are used explicitly.