Multi class segmentation training with images where not all classes are annotated

Hey there!
I have a trained segmentation model for multiple classes (like cat, dog, mouse).
Say, I get a new dataset with images where the classes cat and dog are annotated, but the others aren’t - so there may be an image that has a cat, a dog and a mouse, but only cat & dog are annotated.
I want to train the model further with this dataset.

From my understanding, the model which is already trained for all classes, will still predict the mouse but receive negative feedback because in the mask is no mouse annotated.
This results in inaccurate training, if I got that right.

How can I solve this?

Hi, yes i suppose you will have to use the trained model to make predictions and use the predictions as annotations and correct these annotations before training again.

Sorry for not being clear; I dont want to predict the images in the dataset but use them to train the model.

To be precise, I have a cross-trained DeepLabV3 model that I cross-trained with a dataset with multiple classes.
Now I want to keep training the model with another dataset, but my problem is that in the images of the new dataset, not all classes that my model is trained for are annotated.
I want to use this dataset anyway for training, without making results for the classes that are not annotated worse.

I hope it’s more clear now!

As it is a supervised training i think the ground truth is required if you do not have them you will have to either annotate or go for an unsupervised approach using the features that you can obtain from your pretrained model and try some regional segmentation based on that.

I hope you find an answer by someone just trying to help.

1 Like