loss_fn = reg_BCELoss(dim=2)
loss = 0
def train(...loss_fn):
...
for i_batch, (samples, labels) in enumerate(TrainDL):
...
loss_batch = loss_fn(labels_pred, labels)
loss += loss_batch.item()
...
Now I’d like to set weights inside my training loop but apparently only the constructor can set weights which means, I’d have to create the loss function object inside the loop. Which means, I’d create thousands of objects.
So, what do I do? Do I try to do something stupid?
the weights in the BCELoss could typically be determined before you start your training loop because it usually depends on the label distribution in the whole training data and not in each minibatch.
What do you mean by class? In any case: I don’t want to pass the weights to the constructor but to the forward function. See argument in my other post. The reason for that is simple: I don’t want to go through GB and GB of data and pass several hundred of megabytes of information to one argument. Just seems wrong.
You can calculate the loss, and before doing the reduction over the batch, multiply it with the scaling factor, and then do the reduction. Here’s something implemented:
This is for sequence though, you probably have BxC and not BxTxC, but I think the idea is similar.
I know this is old news now - but I’m running into this now and feel your pain. I wonder how anyone uses weights with BCELoss successfully? Needing to know the weights a priori for each element of each batch to pass to the constructor is such a weird constraint to have…