Semantic segmentation returning the same prediction for each class?

I’m trying to implement a semantic segmentation model. I’m using deeplabv3_resnet50 with transfer learning. The default version of deeplabv3_resnet50 returns 21 classes.

However, I’ve created a deeplabv3_resnet50 model and tried passing an image in. What I’ve noticed is that the result that the model outputs for the first class is exactly the same as result for the second class, and the third class and so on. This doesn’t make sense, and defeats the purpose of semantic segmentation because you can’t actually segment an image.

This is also using the default implementation of deeplabv3_resnet50, so I’m not sure what’s going wrong or how to fix it.

Does anyone know whether I might have done something wrong?