Weight Gradient Remain zero after backward

[Nikronic (Nikan Doosti)]

Hi,
Actually this type of problem is common and called vanishing gradients which weights do not change as gradients are zero in different epochs. It is hard to find the issue by just knowing the config of network. This post may help you through finding the correct config.

Vanishing gradients

Hello. I have a Unet network for segmentation. Using dice loss the net is learning. But when I use Crossentropy, after one epoch the loss is constant. I ve searched on internet and i ve found a function to check gradients, and after 3 epochs they become 0. Can someone help me with some ideas? I already use Relu, and init the conv2d with xavier.

bests