IndexError: Target 2 is out of bounds

vision
22

@vyaslkv why you put output convolution = 32??

23

I have 20 classes. My output shape is [4,20], target shape is [4], but target contains random classes between 387 and 2745 . So, it raises index error, target is out of bounds. How can I resolve it? I know target elements should be range [0, nb-classes - 1]. but, how can I map it. Please guide me with sample code.

24

Yes, you would need to map the values to the expected range.
One approach would be a manual mapping, where you can define which old class index maps to which new one and create the new target tensor by assigning the desired values.
Another valid approach would be to map new values from [0, nb_classes-1] to the unique old targets:

old_target = torch.randint(300, 320, (15,))
print(old_target)
> tensor([315, 308, 316, 310, 313, 314, 309, 304, 307, 303, 311, 311, 312, 302,
        308])

unique_targets = torch.unique(old_target)
print('unique targets: {}'.format(unique_targets))
> unique targets: tensor([302, 303, 304, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316])

new_target = torch.empty_like(old_target)
for idx, t in enumerate(unique_targets):
    print('replacing {} with {}'.format(t, idx))
    new_target[old_target == t] = idx

> replacing 302 with 0
 replacing 303 with 1
 replacing 304 with 2
 replacing 307 with 3
 replacing 308 with 4
 replacing 309 with 5
 replacing 310 with 6
 replacing 311 with 7
 replacing 312 with 8
 replacing 313 with 9
 replacing 314 with 10
 replacing 315 with 11
 replacing 316 with 12
27

It seems you are currently setting the number of classes to 1 based on this:

NUM_CLASSES = 1  # Total number of classes in the dataset

so I’m unsure what the exact use case is.
In your custom loss function (TverskyCrossEntropyDiceWeightedLoss) you are also applying the softmax in dim1, which would return ones for the pred tensor assuming the channel dimension has also a size of 1.

28

where u this code nbclasses - 1give me detail aboutt hat

29

no no.
It also depend on your label types, if it include label ‘0’, you right.

30

噢噢噢
i know what you mean, hahahah, it is same to my think

32

Hi,

I have manually defined class labels mapping as
id2label={0: ‘O’, 1: ‘B-HEADER’, 2: ‘I-HEADER’, 3: ‘B-QUESTION’, 4: ‘I-QUESTION’, 5: ‘B-ANSWER’, 6: ‘I-ANSWER’}
label2id = {‘O’: 0,‘B-HEADER’: 1,‘I-HEADER’: 2,‘B-QUESTION’: 3,‘I-QUESTION’: 4,‘B-ANSWER’: 5,‘I-ANSWER’: 6}

the labels values is list of labels from 0 to 6 , which is [0, nbclasses-1]
however, I am getting error
IndexError: Target 3 is out of bounds.

my input tensors shape to layoutlmv3tokenclassification problem is
torch.Size([2, 512]), torch.Size([2, 512]), torch.Size([2, 512, 4]), torch.Size([2, 512]), torch.Size([2, 3, 224, 224])

where batch_size is 2, 512 is padded length

Please guide, what I am missing?

33

the labels generated from tokenised after padding is
labels: tensor([-100, 0, -100, -100, 3, 3, -100, -100, 3, -100, 5, -100,
3, 3, 0, -100, -100, -100, 1, -100, -100, 2, -100, -100,
2, -100, -100, 2, -100, -100, -100, -100, -100, 2, -100, 2,
3, 4, 4, 4, -100, 4, 5, -100, 6, -100, 6, -100,
6, -100, 6, -100, 6, 5, -100, 6, -100, 6, 3, -100,
-100, 4, -100, 5, -100, 6, -100, 6, -100, -100, 3, 4,
4, 5, -100, -100, -100, 6, 6, 6, 6, 6, 6, 6,
6, 6, -100, 6, 6, 6, -100, 6, 6, 6, 6, 6,
6, 6, -100, -100, 6, 6, -100, -100, 6, 6, 6, 6,
6, 6, 6, 6, 6, 6, 6, 6, -100, -100, 6, 6,
6, 6, 6, 6, 6, 6, 6, 6, 6, -100, 6, 6,
6, 6, 6, 6, 6, 6, 6, 3, -100, 4, 4, -100,
-100, 4, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6,
6, 6, 6, 6, 6, -100, 6, 6, 6, 6, 6, 6,
6, 6, 6, 6, 6, 6, 6, 6, -100, 1, 2, 2,
2, 2, -100, 2, -100, 3, 4, -100, 5, -100, 6, 6,
6, -100, 6, 6, 5, 6, 6, 6, 6, -100, 0, -100,
0, 0, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
-100, -100, -100, -100, -100, -100, -100, -100])

34

the -100 gets automatically ignored for padded tokens?