How to fix my cuda error?

Hi.
I am Mikhail.
I am training my model but error is occurred as follows:
C:/w/1/s/tmp_conda_3.7_104508/conda/conda-bld/pytorch_1572950778684/work/aten/src/THCUNN/SpatialClassNLLCriterion.cu:104: block: [0,0,0], thread: [903,0,0] Assertion t >= 0 && t < n_classes failed.
THCudaCheck FAIL file=…\aten\src\THC\THCCachingHostAllocator.cpp line=278 error=710 : device-side assert triggered
File “c:\Users\OCR Labs\Documents\ERFNet-CULane-PyTorch\train_erfnet.py”, line 173, in train
loss_exist = criterion_exist(output_exist, target_exist_var)
File “C:\Users\OCR Labs\Anaconda3\envs\torch-env\lib\site-packages\torch\nn\modules\module.py”, line 541, in call
result = self.forward(*input, **kwargs)
File “C:\Users\OCR Labs\Anaconda3\envs\torch-env\lib\site-packages\torch\nn\modules\loss.py”, line 601, in forward
reduction=self.reduction)
File “C:\Users\OCR Labs\Anaconda3\envs\torch-env\lib\site-packages\torch\nn\functional.py”, line 2116, in binary_cross_entropy_with_logits
return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
RuntimeError: CUDA error: device-side assert triggered
I am going to get 5 classes as a output class, so i have set 5 class.
My error is occured in following command line.
loss = criterion(torch.nn.functional.log_softmax(output, dim=1), target_var)
here output tensor is tensor([[0.4826, 0.4970, 0.4989, 0.4563, 0.5411]]) and target_var is tensor([[1., 1.,1.,1.,1.]])
How can i fix this issue? Thank you very much!

I think you need to use
torch.sigmoid with binary_cross_entropy
or use binary_cross_entropy_with_logits

if you use log_softmax(), then it will give values less than minus one, for example,

output = torch.tensor([[0.4826, 0.4970, 0.4989, 0.4563, 0.5411]])
torch.log_softmax(output, dim=1)

gives

tensor([[-1.6224, -1.6080, -1.6061, -1.6487, -1.5639]])

as

torch.softmax(output, dim=1)

gives

tensor([[0.1974, 0.2003, 0.2007, 0.1923, 0.2093]])

and if you take log of these values then, you will get output of torch.log_softmax, for example,

import math
math.log(0.1974)

gives

-1.6225231519827559

if you use binary_cross_entropy_with_logits, with above values, then it would be like,

torch.binary_cross_entropy_with_logits(tensor([[-1.6224, -1.6080, -1.6061, -1.6487, -1.5639]]), target)

which gives error, as formula for binary_cross_entropy_with_logits has log(input), and log of negative numbers is not defined.

if you use output directly, for example,

torch.binary_cross_entropy_with_logits(output, target)

then you will get

tensor([[0.4807, 0.4752, 0.4745, 0.4908, 0.4588]])

or if you use

criterion = nn.BCELoss(reduction='none')
criterion(torch.sigmoid(output), target)

then you will get

tensor([[0.4807, 0.4752, 0.4745, 0.4908, 0.4588]])

maybe this is what you are looking for.
or if you do not specify

reduction='none'

then it will take mean of these values by default and give loss.

BCEwithLogitsLoss, uses log-sum-exp, which makes computing loss faster as compared to using sigmoid and then BCELoss.