Multiclass semantic segmentation giving zero in all output channels except for the first channel

Remove it so your model is returning raw logits.