# How can i introduce weights to my final loss

hello,
how can i introduce weights to my final loss and how can i learn the best weight?

``````def criterion(triplet_y, softmax_y, labels):
losses = [embedding_criterion(output, labels) for output in triplet_y] + \
[xent_criterion(output, labels) for output in softmax_y]
loss = sum(losses)
return loss``````

If you use trainable weights on your last loss, the training would push them to a negative value, as it’s yielding the minimal loss, wouldn’t it?

Anyway, you could define a new parameter as:

``````weight = nn.Parameter(torch.tensor(1))
``````

and push it to the optimizer with all other parameters:

``````optimizer = torch.optim.SGD(list(model.parameters()) + [weight], lr=1e-3)
``````

maybe i did not explain well what i want to do
im mean that i wanna have a weight for every loss like this :

`````` losses = W1 *  [embedding_criterion(output, labels) for output in triplet_y] + \
W2 *  [xent_criterion(output, labels) for output in softmax_y]
``````

but this is not working, it generates an error

What kind of error is it generating?

i did

``````0.2 *  [xent_criterion(output, labels) for output in softmax_y] + \
[xent_criterion(output, labels) for output in softmax_y]
``````

the error is
[xent_criterion(output, labels) for output in softmax_y] +
TypeError: can’t multiply sequence by non-int of type ‘float’

but i think that the problem is solved, i must do[ 0.2 * xent_criterion(output, labels) for output in softmax_y] + \

but still dont know how to learn the best valur of the weight, i just take 0.2 randomly

Ah OK, yes the error is thrown as you cannot multiply a scalar with a `list` directly.

I’m still unsure, if “learning” the weight would work, since it would just push it to a negative value.
However, if you want to use the weight as a trainable parameter, refer to my code snippet from my previous post.

1 Like

thank you for your help