Data augmentation changed the mask in semantic segmentation

vision
1

Hi all
I am working on a semantic segmentation problem using satellite image
I want to ask about data augmentation (offline or on the fly ) when I apply it like adding noise or change the contrast so we apply it to both image and mask but in this case, the values of the mask pixels (pixel-wise class value ) changed (ex: value 1 change to 1.5 or something like that ) so, is this a problem? if so, how can I solve it?>

2

any answer guys !!!

3

I guess you are resizing the mask with an interpolation method other than NEAREST, which would interpolate the output values and would thus create new “target” values.
So yes, I think this might be a problem assuming you are working on a multi-class segmentation use case and you should make sure that a nearest neighbors interpolation method is used.

PS: I would recommend to have a bit more patience, as I wouldn’t expect many users to be active on a Sunday and bumping after 3h of creating the topic might not have the desired effect.

1 Like
4

Thank you so much
I am sorry for that

so, besides interpolation what is the best augment method that can I apply to image and mask without changing the mask values?

i used this methods:

train_transform = A.Compose(
[ A.PadIfNeeded(min_height=512, min_width=512, border_mode=4),
A.Resize(512, 512),
A.RandomBrightnessContrast(brightness_limit=0.3,
contrast_limit=0.3, p=0.5),
A.HorizontalFlip(p=0.5),
A.VerticalFlip(p=0.5),
A.MedianBlur(blur_limit=3, always_apply=False, p=0.1),
])

are these methods ok ?

5

No, I don’t think RandomBrightnessContrast and MedianBlur would make sense to be applied on a mask containing class indices as both transformations would change the “pixel values”, which represent class indices (not color values as in e.g. RGB input images).

6

so, which transform operations can I apply to my train dataset?

7

Spatial transformations applied on the input image should also be applied on the mask tensor to make sure that the input pixel location still correspond to the mask (e.g. rotaions, resizing etc.).
Besides that you should treat the mask as a classification target and should not change its values.

1 Like
8

yes, thank you very much
so if i apply the add noise or bullring or something like that operation just to image without mask , is this right?

9

I found this answer in Albumentations website:
Many non-spatial transformations like, [RandomBrightness], [RandomContrast] [RandomGamma] can be also added. They will be applied only to the image and not the mask.