# One of the class merged with Ignore class while training semantic segmentation **URL:** \ **Category:** vision\ **Created:** [January 10, 2020, 12:59pm UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289 "2020-01-10T12:59:11Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Shashank\_Shekhar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shashank_shekhar/32/12704_2.png) [@Shashank\_Shekhar](https://discuss.pytorch.org/u/Shashank_Shekhar)\ **Post date:** [January 10, 2020, 12:59pm UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/1 "2020-01-10T12:59:11Z")
While training semantic segmentation i had 23 classes including Ignore label, during my prediction i get only 22 classes. I have used **cross entropy loss** and provided **ignore index as 255**. What looks from the predicted image is one of the class **(class\_name:vegetation, id:21)** merge with ignore index as seen in the below image. **Below image shows the Unique values for Ground truth and Prediction along with Original, GT & Predicted image.** Please help with what may have caused this!!! ![Screenshot from 2020-01-10 18-08-21](https://discuss.pytorch.org/uploads/default/original/3X/7/a/7a8b44a08f9f1b78e4e78213f4665cce28737118.png) ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 10, 2020, 11:27pm UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/2 "2020-01-10T23:27:41Z")
What is the shape of your output activation? In a segmentation use case, your output activation will most likely have the shape `[batch_size, nb_classes, height, width]`. If `nb_classes` was defined as 22, you will only get these labels. ---
**Author:** ![Shashank\_Shekhar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shashank_shekhar/32/12704_2.png) [@Shashank\_Shekhar](https://discuss.pytorch.org/u/Shashank_Shekhar)\ **Post date:** [January 11, 2020, 1:59am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/3 "2020-01-11T01:59:46Z")
Thanks for replying @ptrblck , I am using deeplab v3+ architecture and my segmentation output shape is in **[batch\_size, nb\_classes, height, width]** format. I just printed out the unique values to check whether ignore index is their or not. During training output tensor is of shape [batch, 23, 512, 512] after that i calculated the labels using `_, pred = torch.max(scores, dim=1)` .List of unique values of pred contain **22 (which is ignore index)**, But at inference time when i use the trained weights, output is of shape [batch, 23, 512, 512] and the the unique values contains **max label upto 21**. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 11, 2020, 2:03am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/4 "2020-01-11T02:03:05Z")
It seems that the model output contains the background class + all other 22 classes. If I understand your work flow correctly, you have passed the background class index as `ignore_index`, so that your model never learned how to predict the background? If that’s the case, I would expect the model never to output the background class during inference either. ---
**Author:** ![Shashank\_Shekhar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shashank_shekhar/32/12704_2.png) [@Shashank\_Shekhar](https://discuss.pytorch.org/u/Shashank_Shekhar)\ **Post date:** [January 11, 2020, 5:25am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/5 "2020-01-11T05:25:40Z")
FYI i am using appoloscapes dataset, below are the labels available and in the trainID column **we don’t have any classes for background**. So i am using **ignore index + 22 other classes**. For the architecture, loss i am passing below argument: ```auto model = DeepLab (num_classes=23,backbone=resnet101,output_stride=16) criterion = nn.CrossEntropyLoss(ignore_index=255) ``` ```auto labels = [ # name clsId id trainId category catId hasInstanceignoreInEval color Label('others' , 0 , 0, 255 , '其他' , 0 ,False , True , 0x000000 ), Label('rover' , 0x01 , 1, 255 , '其他' , 0 ,False , True , 0X000000 ), Label('sky' , 0x11 , 17, 0 , '天空' , 1 ,False , False , 0x4682B4 ), Label('car' , 0x21 , 33, 1 , '移动物体', 2 ,True , False , 0x00008E ), Label('car_groups' , 0xA1 , 161, 1 , '移动物体', 2 ,True , False , 0x00008E ), Label('motorbicycle' , 0x22 , 34, 2 , '移动物体', 2 ,True , False , 0x0000E6 ), Label('motorbicycle_group' , 0xA2 , 162, 2 , '移动物体', 2 ,True , False , 0x0000E6 ), Label('bicycle' , 0x23 , 35, 3 , '移动物体', 2 ,True , False , 0x770B20 ), Label('bicycle_group' , 0xA3 , 163, 3 , '移动物体', 2 ,True , False , 0x770B20 ), Label('person' , 0x24 , 36, 4 , '移动物体', 2 ,True , False , 0x0080c0 ), Label('person_group' , 0xA4 , 164, 4 , '移动物体', 2 ,True , False , 0x0080c0 ), Label('rider' , 0x25 , 37, 5 , '移动物体', 2 ,True , False , 0x804080 ), Label('rider_group' , 0xA5 , 165, 5 , '移动物体', 2 ,True , False , 0x804080 ), Label('truck' , 0x26 , 38, 6 , '移动物体', 2 ,True , False , 0x8000c0 ), Label('truck_group' , 0xA6 , 166, 6 , '移动物体', 2 ,True , False , 0x8000c0 ), Label('bus' , 0x27 , 39, 7 , '移动物体', 2 ,True , False , 0xc00040 ), Label('bus_group' , 0xA7 , 167, 7 , '移动物体', 2 ,True , False , 0xc00040 ), Label('tricycle' , 0x28 , 40, 8 , '移动物体', 2 ,True , False , 0x8080c0 ), Label('tricycle_group' , 0xA8 , 168, 8 , '移动物体', 2 ,True , False , 0x8080c0 ), Label('road' , 0x31 , 49, 9 , '平面' , 3 ,False , False , 0xc080c0 ), Label('siderwalk' , 0x32 , 50, 10 , '平面' , 3 ,False , False , 0xc08040 ), Label('traffic_cone' , 0x41 , 65, 11 , '路间障碍', 4 ,False , False , 0x000040 ), Label('road_pile' , 0x42 , 66, 12 , '路间障碍', 4 ,False , False , 0x0000c0 ), Label('fence' , 0x43 , 67, 13 , '路间障碍', 4 ,False , False , 0x404080 ), Label('traffic_light' , 0x51 , 81, 14 , '路边物体', 5 ,False , False , 0xc04080 ), Label('pole' , 0x52 , 82, 15 , '路边物体', 5 ,False , False , 0xc08080 ), Label('traffic_sign' , 0x53 , 83, 16 , '路边物体', 5 ,False , False , 0x004040 ), Label('wall' , 0x54 , 84, 17 , '路边物体', 5 ,False , False , 0xc0c080 ), Label('dustbin' , 0x55 , 85, 18 , '路边物体', 5 ,False , False , 0x4000c0 ), Label('billboard' , 0x56 , 86, 19 , '路边物体', 5 ,False , False , 0xc000c0 ), Label('building' , 0x61 , 97, 20 , '建筑' , 6 ,False , False , 0xc00080 ), Label('bridge' , 0x62 , 98, 255 , '建筑' , 6 ,False , True , 0x808000 ), Label('tunnel' , 0x63 , 99, 255 , '建筑' , 6 ,False , True , 0x800000 ), Label('overpass' , 0x64 , 100, 255 , '建筑' , 6 ,False , True , 0x408040 ), Label('vegatation' , 0x71 , 113, 21 , '自然' , 7 ,False , False , 0x808040 ), Label('unlabeled' , 0xFF , 255, 255 , '未标注' , 8 ,False , True , 0xFFFFFF ), ] ``` ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 11, 2020, 5:51am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/6 "2020-01-11T05:51:23Z")
Thanks for the information. Could you check, what the white color represents in your ground truth image? I would assume it’s class0 (sky), but that wouldn’t make sense, as part of the buildings and car is also marked as this color. ---
**Author:** ![Shashank\_Shekhar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shashank_shekhar/32/12704_2.png) [@Shashank\_Shekhar](https://discuss.pytorch.org/u/Shashank_Shekhar)\ **Post date:** [January 11, 2020, 8:59am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/7 "2020-01-11T08:59:23Z")
what you told is correct ,actually all the white color in the image is ignore index. Sky, building and host vehicle all should have corresponding classes but the dataset provided by them labels some parts as 255. Is their any workaround? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 12, 2020, 1:36am UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/8 "2020-01-12T01:36:59Z")
This would explain, why your model predicts random classes for these pixel locations. Basically, your model will not get any information for these locations, as they are ignored, and can just output any random class. In your case it seems the vegetation class was picked. You could transform all 255 class values to an “unknown” or background class and let the model learn this additional class instead. ---
**Author:** ![Shashank\_Shekhar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shashank_shekhar/32/12704_2.png) [@Shashank\_Shekhar](https://discuss.pytorch.org/u/Shashank_Shekhar)\ **Post date:** [January 13, 2020, 1:09pm UTC](https://discuss.pytorch.org/t/one-of-the-class-merged-with-ignore-class-while-training-semantic-segmentation/66289/9 "2020-01-13T13:09:00Z")
It worked, i trained my model for 10 epoch by adding all the ignore labels to extra class(class id:22). I have attached the results i got after 10 epochs. Thanks @ptrblck for your support, i have closed the issue with your last answer. ![Screenshot from 2020-01-13 18-32-21](https://discuss.pytorch.org/uploads/default/original/3X/2/3/23f7077cf7e173bd698a8e825d8c95da90c996ac.png)