My code is like that
class Net(nn.Module):
def __init__(self, upscale_factor):
super(Net, self).__init__()
# self.activation = nn.ReLU()
# self.gain = "relu"
self.activation = nn.Tanh()
self.gain = "tanh"
self.image_conv1 = nn.Conv2d(5, 20, 5, padding=2)
self.image_conv2 = nn.Conv2d(20, 20, 5, padding=2)
self.low_conv1 = nn.Conv2d(5, 15, 3, padding=1)
image_feature_1 = self.activation(self.image_conv1(image))
image_feature_2 = self.activation(self.image_conv2(image_feature_1))
When I use Tanh function, it throw the error
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
However, when I use relu function, everything works well.
Is there any difference between these two activation function?