Hi @ptrblck
I just added a bit of debugging code in my dataset generator, to see if the masks are being sent to the model correctly.
mask = self.mask_generator.mask(id=id_, height=h, width=w)
# TODO: Remove this code
import matplotlib.pyplot as plt
# display individual mask channels
for i in range(mask.shape[-1]):
mask_ch = mask[..., i] * 255
plt.figure(figsize=(10, 10))
plt.title('mask ch{}'.format(i + 1))
plt.imshow(mask_ch, cmap=plt.cm.gray)
plt.show()
These are how the individual channel masks look like, corresponding to builds, roads, etc. There are 10 classes in total, which have been mapped to 10 channels in the generated mask. Here are the first 3 masks:
I just took a look at the labels_batch, and after unsqueezing it and trying to display the first channel, I see that the image contains a sum of the individual classes and it is inverted and a fully concatenaed negative mask!
So, the neural network doesn’t learn anything for the individual channels because the target labels are all concatenated for the individual channels.
Let me debug this, and make sure the class to index tensor mapping operation and subsequent recovery of the original data is correctly working.