The Kumaraswamy distribution is defined over the open interval (0,1) but torch.distributions.kumaraswamy.Kumaraswamy can generate samples that equal 1. This causes “nan” outputs when computing the log_prob of such samples.
import torch as ch
import torch.distributions as dist
kumaraswamy = dist.kumaraswamy.Kumaraswamy(ch.Tensor([0.2]), ch.Tensor([0.2]))
samples = kumaraswamy.sample(ch.Size([1000000]))
assert len(samples[samples==1.0]) > 0
Since torch.distributions.uniform.Uniform can generate samples that equal 0, these are perhaps transformed to samples that equal 1 for the Kumaraswamy distribution, outside its support.
This doesn’t seem likely because your test with “only” 1,000,000 samples is
quite unlikely to sample an exact 0.0 from Uniform.
Note that floating-point numbers can get closer to 0.0 than to 1.0. I expect
that samples from Uniform that are close to 0.0 (but not equal) get transformed
to 1.0 (and Uniform samples that are close to 1.0 get transformed to 0.0).
Thanks for pinging on this issue!
I assume this Uniform usage might create these invalid values? @kiranchari As @KFrank mentioned, would you mind creating an issue on GitHub so that we could track and fix it, please?
I found the same bug and lately discovered, that this topic was already open.
What is the status?
I’m using pytorch 2.5 and the Kumaraswamy distribution still has that bug: sampling from it I can get 0, which should not be.
It seems that I’m not the only one experiencing problem with the pytorch version of the kumaraswamy distribution. I found this paper: https://arxiv.org/pdf/2410.00660 and they use the exact pytorch version, which is known to be unstable, for their study.
That is my paper, it’s linked in the original pull request I posted above. We find all common implementations have unresolved numerical issues in the relevant distribution-related functions used for probabilistic inference. We provide a stable Kumaraswamy implementation, which I’m trying to integrate into PyTorch.
I did not see, that your name here the same is on the paper.
But you are g**damn right. KM is strangly under-unused.
By the way, I couldn’t find the code you mention in the paper, because there is not link to any repository. In the link you posted above, I’m not sure how to get to right place.