You could use forward hooks to get the desired activation. Here is a small example for a dummy network.