I have a program with a sub neural network whose forward
method takes three input arguments:
def forward(self, input, hidden,
transformed_input=None):
But in reality it was fed with the latter two arguments, namely hidden
and tranformed_input
.
I registered one forward hook and found only two inputs have been captureds:
def hook(module, input, output):
print(len(input)) #it outputs 2
When I am trying to print input
in my forward hook:
def hook(module, input, output):
#print(module)
print(input)
It outputs a tuple of size 2 with (None, (some tensor, some tensor)).
Notice in my program hidden
does have this kind of (some tensor, some tensor) shape. My best guess so far is that PyTorch ignores some parts whose default value is None. Since in my program in the forward pass input
was fed with None and transformed_input
's default vallue is None, hence causing the above observation.
However, I did another experiement:
class Test(nn.Module):
def __init(self):
pass
def forward(self, input1, input2, input3=None):
if input1 is not None:
return input1+input2
else:
return input2
a=Test()
def hook(module, input, output):
print(module)
print(len(input))
print(input)
a.register_forward_hook(hook)
a(None, torch.tensor([1]), torch.tensor([9]))
# Test()
# 3
# (None, tensor([1]), tensor([9]))
#tensor([1])
which correctly output everything. So what are your thoughts on this?