I am beginner in PyTorch and want to create something like:
class net()
…
…
…
def forward(self, x):
out = self.layerA(x)
out = self.layerB(out)
pLoss = calcPLoss(out)
out = self.layerC(out)
return out, pLoss
train()
…
out, pLoss = net(input)
loss = criterion(out, input)
loss = loss + pLoss
loss.backward()
Here, I have two question:
I am not able to return multiple variables from forward function. Can somebody show me correct way to do this?
If I would be able to do this, does PyTorch is able to manage the gradients accordingly? Because my pLoss is calculated in the intermediate layer and I have directly summing up this with the final layer loss. But I want my gradients from pLoss to backpropogate only before intermediate layer.
I am getting error like “if input is not None and input.dim() != 4:
AttributeError: ‘tuple’ object has no attribute ‘dim’” when I try to return those two values.
When I won’t return pLoss and just return the calculated value. It won’t throw any error.
Are you sure that you didn’t made a typo and pass to the criterion the tuple of the two outputs instead of only out?
You may want to print the returned values to make sure they are what you think.
There are no problem returning multiple elements.