# Initializing single number for re-summing in autograd?

Hi, I’m interested in trying to combine loss criteria, and I made a little function, to try to do that. Here’s what DOESN’T work…

``````def calc_loss(Y_pred, Y_true, criteria, lambdas):
#  Y_pred: the output from the network
#  Y_true: the target values
#  criteria: an iterable of pytorch loss criteria, e.g. [torch.nn.MSELoss(), torch.nn.L1Loss()]
#  lambdas:  a list of regularization parameters
assert len(criteria) == len(lambdas), "Must have the same number of criteria as lambdas"
loss = torch.zeros(1, requires_grad=True)       # not sure how to properly initialize a single pytorch number for autograd
for i in range(len(criteria)):
loss += lambdas[i] * criteria[i](Y_pred, Y_true)
return loss
``````

When I do that, I get the Runtime error at the “loss +=” line…

``````RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.
``````

So instead, here’s the working routine that I have right now, but it’s just not ‘elegant’ IMHO:

``````def calc_loss(Y_pred, Y_true, criteria, lambdas):
assert len(criteria) == len(lambdas), "Must have the same number of criteria as lambdas"

for i in range(len(criteria)):
if (0==i):
loss = lambdas[i] * criteria[i](Y_pred, Y_true)
else:
loss += lambdas[i] * criteria[i](Y_pred, Y_true)
return loss
``````

For future reference how should you properly initialize a ‘zero’ loss of this type?
(I’m using Pytorch 0.4)

Thanks!

PS- Also tried a simple…

``````    loss = torch.dot(lambdas, criteria(Y_pred, Y_true))
``````

…but that was just wishful thinking, because the lambdas are ‘just numbers’, so you get `TypeError: 'list' object is not callable`. Even if you wrap the lambdas in torch.tensor(), you get the same error.

Hi,

You can just initialize it with a python `0`.
I haven’t run it, but the following code should work ``````def calc_loss(Y_pred, Y_true, criteria, lambdas):
#  Y_pred: the output from the network
#  Y_true: the target values
#  criteria: an iterable of pytorch loss criteria, e.g. [torch.nn.MSELoss(), torch.nn.L1Loss()]
#  lambdas:  a list of regularization parameters
assert len(criteria) == len(lambdas), "Must have the same number of criteria as lambdas"
loss = 0
for i in range(len(criteria)):
loss += loss lambdas[i] * criteria[i](Y_pred, Y_true)
return loss
``````

Wow, that’s so simple. I guess I’d anticipated getting ‘Expected __Tensor but got ___Tensor’ errors if I tried that!
Thanks. Yes, it works.