My code is like that
class Net(nn.Module): def __init__(self, upscale_factor): super(Net, self).__init__() # self.activation = nn.ReLU() # self.gain = "relu" self.activation = nn.Tanh() self.gain = "tanh" self.image_conv1 = nn.Conv2d(5, 20, 5, padding=2) self.image_conv2 = nn.Conv2d(20, 20, 5, padding=2) self.low_conv1 = nn.Conv2d(5, 15, 3, padding=1) image_feature_1 = self.activation(self.image_conv1(image)) image_feature_2 = self.activation(self.image_conv2(image_feature_1))
When I use Tanh function, it throw the error
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
However, when I use relu function, everything works well.
Is there any difference between these two activation function?