If someone comes by this via Google search, I wanted to update why I was getting this loss behaviour. The reason was that I was applying nn.Sigmoid() at the model output and in the DiceLoss() as well. Removing the sigmoid operation from the model output fixed the weird loss behaviour.
ptrblk explained this more here: Why my loss function's value doesn't going down? which helped me fix this.