Speed of a self-made algorithm layer in Pytorch

Are you applying torch.exp on the inputs before? If not, dividing by the sum alone shouldn’t yield the same output as the softmax.
However, as I’m not familiar with your use case, it’s good to hear you can use built-in methods. :wink: