I am trying to code a test case for multi task learning, using my own loss function. The idea is that the output layer is 3-dimensional, the first output is used for 1D regression, the last two are used for 2-class classification. So, my combined loss function is a weighted sum of L1 loss and CELoss.
However, pytorch is complaining about my datatypes?
class my_loss:
def __init__(self,weights):
self.weights=weights
#age loss:
self.L1=nn.SmoothL1Loss(reduction='mean',beta=0.05)
#gender loss:
self.CE=nn.CrossEntropyLoss(reduction='mean')
def __call__(self,output,target):
loss = self.weights[0]*self.L1(output[:,0],target[:,0])+self.weights[1]*self.CE(output[:,1:3],target[:,1])
return loss
#testing:
it=iter(dataloaders_dict['train'])
X,y=it.next()
criterion = my_loss(weights=(1,1))
out=model_ft(X)
loss=criterion(out,y)
print(y.dtype)
print(X.dtype)
print(out.dtype)
print(loss.dtype)
loss.backward()
This returns:
torch.int64
torch.float32
torch.float32
torch.float32
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-88-fc96d4e3072b> in <module>()
31 print(loss.dtype)
32
---> 33 loss.backward()
1 frames
/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)
145 Variable._execution_engine.run_backward(
146 tensors, grad_tensors_, retain_graph, create_graph, inputs,
--> 147 allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
148
149
RuntimeError: Found dtype Long but expected Float
If only my targets are long, how come loss.backward is finding long’s, and how do I fix it?