How to fix the “CUDA error: device-side assert triggered” error?

1

I have segmentation labels [0, 1, 2], where class label 2 is invalid region, so I want to ingore that class id. I build the model as

num_classes = 2
model = SegModel(num_classes=2, ...)
loss = LossMethod(..., ignore_class_ids=2)
model.train(...)

I pass the gt value as it is (with unique gt value [0,1,2]) and loss method have this information which class id to ignore. And for that, number of classes are set to 2 (for [0, 1]).

Maybe for this setup, I got the following error

RuntimeError: CUDA error: device-side assert triggered

Is such modelling approach invalid in torch? Or there are other catch?

2

Check which operation raises this error as it seems you are not masking the invalid índices properly.