Is there an expected distribution for torch.empty?

1

I was reviewing the gumbel_softmax implementation [1] and realized that the Gumbel distribution was being sampled via gumbels = -torch.empty_like(logits, memory_format=torch.legacy_contiguous_format).exponential_().log() # ~Gumbel(0,1). I couldn’t see how this would generate gumbel = -log(eps - log(uniform(0,1) + eps) so I empirically validated the distributions which checks out.

My question is the following: what is the distribution of torch.empty so that these formula generate the same distribution?

[1] https://pytorch.org/docs/stable/_modules/torch/nn/functional.html#gumbel_softmax

2

it is garbage rewritten by an inplace operation (here exponential_() creates samples from Exponential(1) and is not a function of input data)

1 Like