How are probs from logits computed in Categorical?

1

Hello everybody!

Just by looking at the code of Categorical and its base class, I was not able to find a line that assigns something to self.probs, if logits are supplied as the only argument. While using the debugger to step through the code, I observed that self.probs is assigned magically once the line

self.logits = logits - logits.logsumexp(dim=-1, keepdim=True)

categorical.py#L60

is run.

It would be great if someone could shed some light onto this matter!

Thanks

2

There is something wrong with the URL you posted.

3

The link is fixed now.

I think I know the answer now.

It is due to the lazy property.

4

For those stumbling across this issue, I found The Log-Sum-Exp Trick to be very helpful.