I’m learning the usage of torch.nn.NLLLoss() and torch.nn.LogSoftmax(), and I’m confused about the results of them.
lsm = torch.nn.LogSoftmax(dim=-1) nll = nn.NLLLoss() grnd_truth = torch.tensor() # let's say it predicted correctly!!! raw_logits = torch.tensor([[0.3665, 0.5542, -1.0306]]) logsoftmax = lsm(raw_logits) final_loss = nll(logsoftmax, grnd_truth) logsoftmax: >>> tensor([[-0.8976, -0.7099, -2.2947]]) final_loss: >>> tensor(0.7099) # Is it normal getting so large loss when predicted correctly? # let's try a incorrect prediction grnd_truth = torch.tensor() nll(lsm(raw_logits), grnd_truth) >>> tensor(0.8976) # Is this not enough large?
Obviously, we can see that the mis-classified loss is not large enough, comparing to the loss of a correctly classifing that is near to former.
I read the docs of NLLLoss, and I know its formula. I guessed that the designers must have their reasons to use the formula, but I didn’t have gotten it.
In the fact, the reason of I posting this question, is that I encountered a difficulty to train a triple-classification module with NLLLoss() and LogSoftmax().
Is the goal of a lossing func that make the loss as small as possible when the prediction is right, and vice versa?