# Passing custom weights to cross entropy loss

I am trying to assign different weights to different classes, so I have modified my loss criterion as such:

I had to convert the weight tensor to double `torch.DoubleTensor(weight)` since my `model` is already moved to `double()`.

Am I doing this correctly ?

``````weights = [0.4,0.8,1.0]

class_weights = torch.DoubleTensor(weights).cuda()

criterion = nn.CrossEntropyLoss(weight=class_weights)``````

This should be correct, yes. Are you seeing any issues using this code?

1 Like

thanks, I just wanted to confirm, the code works just fine.

Hi @ptrblck, related to this question:

How can we be sure that weights are mapped to the proper class? During training, the dataset will be in batch (and shuffled). Should I map weights according to how the model is seeing each label in each batch?

I’m asking that because I would be much more comfortable passing a dictionary of weights to the loss function, instead of a list

In my understanding, classes are assigned with integers (from 0 to `num_classes`).
Following this, weights will be a list of `num_classes`, with `weights[i]` corresponding to weight of `i`th class.

So if I understood correctly, if I have a situation like this:

``````labels = [2, 2, 1, 3]
``````

I calculate weights

``````weights = [0.2, 0.2, 0.4, 0.4]
``````

but to the loss functions I should pass them as

``````weights = [0.4, 0.2, 0.4, 0.4]
``````

because classes are assigned from 0 to num_classes, so `[1, 2, 2, 3]`

Am I right?

Can you elaborate a bit on how you calculate the weights?

Also, both the `weights` lists are different here.

Basically, the `weights` list has length = number of total classes. (Not the number of samples)
`weights[i]` = weight calculated for class `i`.
I am not sure how your example relates to this definition. Maybe I am a bit confused with your example.

You are totally right, let me explain better now (and correct my mistakes)

For example, image I have a dataframe `df` like this

``````| var | label
|  a  | 2
|  b  | 2
|  c  | 1
|  d  | 3
``````

In the example before, I calculated the class weights as

``````w = 1 / (df.label.value_counts()/df.label.value_counts().sum())
w = w/w.sum()
``````

which leads to this class weight (you are right, number total class not number of samples)

``````2    0.2
3    0.4
1    0.4
``````

Now, when I pass `w` to my loss function through `torch.from_numpy(w.values)`, should I first order it so that it reflects the order of the classes (1, 2, 3), therefore `[0.4, 0.2, 0.4]`? Or should I pass it as-is, therefore `[0.2, 0.4, 0.4]`?

Hope it’s clearer now, sorry about before!

Thanks for explaining!
Assuming that you are mapping the classes [1,2,3] into pytorch equivalent 0-based classes of [0,1,2], you have to pass `[0.4, 0.2, 0.4]` as you mentioned.

1 Like