The pytorch docs on the multinomial distribution torch.multinomial( input , num_samples , replacement=False , * , generator=None , out=None) say about the input argument:
input (Tensor) – the input tensor containing probabilities
and they further say:
The rows of
inputdo not need to sum to one (in which case we use the values as weights), but must be non-negative, finite and have a non-zero sum.
Unfortunately, it is not clear what is meant by “weights” and how they are transformed into probabilities, but even so, this shouldn’t be a problem if input sums to 1, so it should be interpreted as the probabilities. Then why does the following happen?
$ torch.multinomial(torch.tensor([0, 1/2, 1/2, 0]), 4)
tensor([2, 1, 0, 3])
Apparently it first fraws the “possible” options, and then due to the default statement “replacement_False”, it moves on to the other two somehow. But I don’t see why options 0 and 3 are drawn at all, since they started out with a probability of 0. I also don’t understand how “replacement=False” works when all that is provided are the probabilities of the different options.
Similarly, according to the docs, torch.multinomial(torch.tensor([0, 10, 3, 0], dtype=torch.float), 4) should result in an error, while in my case, it doesn’t:
$ torch.multinomial(torch.tensor([0, 10, 3, 0], dtype=torch.float), 4)
tensor([1, 2, 0, 3])
So why does this happen?
I’m using pytorch 2.0.0