Hmm, but how does Pytorch implement that integral? Is there no simple function equivalent to Pytorch’s implementation you could provide that I could enter into the following website:
I need to derive an MLP that uses GeLU activations and it would be convenient if I could just enter the full formula into that tool above.