How are probs from logits computed in Categorical?

Hello everybody!

Just by looking at the code of Categorical and its base class, I was not able to find a line that assigns something to self.probs, if logits are supplied as the only argument. While using the debugger to step through the code, I observed that self.probs is assigned magically once the line

self.logits = logits - logits.logsumexp(dim=-1, keepdim=True)

is run.

It would be great if someone could shed some light onto this matter!


There is something wrong with the URL you posted.

The link is fixed now.

I think I know the answer now.

It is due to the lazy property.

For those stumbling across this issue, I found The Log-Sum-Exp Trick to be very helpful.