During training neural network, a loss function looks strange. I used VGG architecture from torchvision, but I modified it to work with BCEloss and CEL ( CrossEntropyLoss) at the same time.
The loss function is now seen as : L = (1-k) BCEloss + k CEL.
Is that possible that most neurons are dead at the beginning of training?
I think that is not related to data or dataloader because I load the same data to ResNet and the loss function looks normal there.
Thanks in advance for suggestions.