Custom activation functions with Learnable Parameter

How can I implement something like PRelu by inheriting autograd.Function?

You likely want nn.Module for holding the parameters, in fact see the PyTorch implementation of nn.PReLU and also the What is torch.nn tutorial.

Best regards

Thomas